This commit fixes an issue where when backtraces were disabled but a
host function returned an error it would trigger a debug assertion
within Wasmtime. The fix here is to update the condition of the debug
assertion and add a test doing this behavior to ensure it works in the
future.
I've also further taken the liberty in this commit to remove the
deprecation notice for `Config::wasm_backtrace`. We don't really have a
strong reason for removing this functionality at this time and users
have multiple times now reported issues with performance that seem
worthwhile to keep the option. The latest issue, #5577, has a use case
where it appears the quadratic behavior is back in a way that Wasmtime
won't be able to detect. Namely with lots of wasm interleaved with host
on the stack if the original error isn't threaded through the entire
time then each host error will trigger a new backtrace since it doesn't
see a prior backtrace in the error being returned.
While this could otherwise be fixed with only capturing one contiguous
backtrace perhaps this seems reasonable enough to leave the
`wasm_backtrace` config option for now.
Closes#5577
* wiggle: Inline some trivial functions
This commit marks a number of functions in wiggle as `#[inline]` as
they're otherwise trivial, mostly returning constants. This comes out of
some work I looked at recently with Andrew where some of these functions
showed up in profiles when they shouldn't.
* wiggle: Optimize the `GuestMemory` for shared memory
This commit implements a minor optimization to the `GuestMemory`
implementation for Wasmtime to skip most methods if a shared memory is
in play. Shared memories never get borrowed and this can be used to
internally skip some borrow-checker methods.
* wiggle: Optimize `GuestPtr::to_vec`
This commit replaces the safe implementation of `GuestPtr::to_vec` with
an unsafe implementation. The purpose of this is to speed up the
function when used with shared memory which otherwise performs a bunch
of atomic reads for types like `u8` which does validation-per-element
and isn't vectorizable. On a benchmark I was helping Andrew with this
sped up the host code enough to the point that guest code dwarfed the
execution time.
* Fix build
Following up on #5535, treat positive and negative zero as inequal in
wasmtime::component::Val::Float{32,64}'s `PartialEq` logic. IEEE 754
equality considers these values equal, but they are semantically
distinct values, and testing and fuzzing should be aware of the
difference.
Update the documentation for `Caller::get_export` to clarify that it's
not expected to be removed in the future. Components do offer an
alternative to `Caller::get_export`, so add a brief note mentioning
that.
Also, as of #4431 `get_export` now works for all exports, not just
memories and functions.
* wasi: avoid buffer underflow with shared memory
This change fixes an issue identified when using wasi-threads to perform
file reads. In order to maintain Rust safety guarantees in the presence
of WebAssembly shared memory, which can be modified concurrently by any
of the running threads, the WASI implementations of `fd_read` and
`fd_pread` were given special code paths when shared memory is detected:
in these cases, the data is first read into a host-limited buffer and
then subsequently copied into linear memory. The problem was that the
rather-complex logic for doing this "buffer then copy" idea for multiple
IO vectors could fail due to buffer underflow. If, e.g., a read was
limited by the host to 64K (or even if the read returned less than the
total buffer size) the `UnsafeGuestSlice::copy_from_slice` logic would
fail, complaining that the sizes of both buffers were unequal.
This change both simplifies and fixes the logic:
- only the first IO vector is filled; this could represent a performance
penalty for threaded programs, but the "buffer then copy" idea already
imposes a non-trivial overhead. This simplifies the logic, allowing us
to...
- resize the shared memory buffer to the exact number of bytes read
* review: early return when no IO vectors passed to shared memory
* fix: add empty RoFlags on early exit
* Wasmtime: Add `Config::disable_cache`
* bench-api: Always disable the cache
* bench-api: Always get a `Config` from CLI flags
This commit fixes an issue that I ran into just now where benchmarking
one `*.so` with `--engine-flags` was giving wildly unexpected results
comparing to something without `--engine-flags`. The root cause here
appears to that when specifying `--engine-flags` the CLI parsing code is
used to create a `Config` and when omitted a `Config::new` instance is
created. The main difference between these is that for the CLI caching
is enabled by default and for `Config::new` it is not. Coupled with the
fact that caching doesn't really work for the `main` branch this ended
up giving wild results.
The fix here is to first always use the CLI parsing code to create a
`Config` to ensure that a config is consistently created. Next the
`--disable-cache` flag is unconditionally passed to the CLI parsing to
ensure that compilation actually happens.
Once applied this enables comparing an engine without flags and an
engine with flags which provides consistent results.
* Fix compile error
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* add clif-util compile option to output object file
* switch from a box to a borrow
* update objectmodule tests to use borrowed isa
* put targetisa into an arc
In #5510 we changed the value types of these variants from u{32,64} to
f{32,64}. One side effect of this change was that two NaN values would
no longer compare equal. While this is behavior complies with IEEE-754
floating point operations, it broke equality assumptions in fuzzing.
This commit changes equality for Val to make NaNs compare equal. Since
the component model requires NaN canonicalization, all NaN bit
representations compare equal, which is different from the original
behavior.
This also gives Vals the semantics of Eq again, so that trait impl has
been reintroduced to related types as well.
This unlocks certain bounds checking optimizations in some
configurations. Wasn't able to measure any delta in sightglass, but still worth
doing anyways.
* wiggle: copy guest strings from shared memory
Along the same lines as #5471, this change adds a new smart pointer,
`GuestStrCow`, to copy the string bytes over from Wasm memory to the
host when the string is found in shared memory. This is necessary to
maintain Rust guarantees: with shared memory, the bytes backing a
`GuestStr` could be altered by another thread and this would invalidate
the assumption that we can dereference at any point to `&str`.
`GuestStrCow` is essentially a wrapper around `GuestStr` when the memory
is not shared but copies the memory region into a `String` when the
memory is shared.
This change updates the uses of Wiggle strings in both wasi-common and
wasi-crypto.
* review: perform UTF-8 check on `GuestStr` construction
The definitions of `wasmtime::component::Val::Float{32,64}` mirrored
`wasmtime::Val::F{32,64}` by using integers as their wrapped types,
storing the bit representation of their floating point values.
This was necessary for the core Wasm `f32`/`f64` types because Rust
floats don't have guaranteed NaN bit representations.
The component model `float32`/`float64` types require NaN
canonicalization, so we can use normal Rust `f{32,64}` instead.
Closes#5480
This change upgrades `UnsafeGuestSlice` in Wiggle to expose more
functionality to be able to use `std::ptr::copy` for writing bytes into
Wasm shared memory. Additionally, it adds a new `GuestCow` type for
delineating between Wasm memory regions that can be borrowed (non-shared
memory) or must be copied (shared memory) in order to maintain Rust
guarantees.
With these in place, it is now possible to implement the `preview1`
"read" functions for shared memory. Previously, these would panic if
attempting to copy to a shared memory. This change removes the panic and
introduces some (rather complex) logic for handling both the shared and
non-shared cases:
- if reading into a Wasm non-shared memory, Wiggle guarantees that no
other guest pointers will touch the memory region and, in the absence
of concurrency, a WASI function can write directly to this memory
- if reading into a Wasm shared memory, the memory region can be
concurrently modified. At @alexcrichton's request re: Rust safety,
this change copies all of the bytes into an intermediate buffer before
using `std::ptr::copy` to move them into Wasm memory.
This change only applies to the `preview0` and `preview1`
implementations of `wasi-common`. Fixing up other WASI implementations
(esp. wasi-crypto) is left for later.
This adds a new error type `UnknownImportError` which will be returned
(wrapped in an `anyhow::Error`) by `Linker::instantiate{,_async,_pre}`
if a module has an unresolvable import.
This error type is also used by `Linker::define_unknown_imports_as_traps`;
any resulting traps will also downcast to `UnknownImportError`.
Closes#5416
This updates the tests to version 0.11 of the wasi bindings. There
aren't any fundamental changes here; this just syncs up with the latest
version so that it's consistent with other users of the wasi APIs.
* Use the `sym` operator for inline assembly
Avoids extra `#[no_mangle]` functions and undue symbols being exposed
from Wasmtime. This is a newly stabilized feature in Rust 1.66.0. I've
also added a `rust-version` entry to the `wasmtime` crate to try to head
off possible reports in the future about odd error messages or usage of
unstable features if the rustc version is too old.
* Fix a s390x warning
* Add `rust-version` annotation to Wasmtime crate
As the other main entrypoint for embedders.
When these engine flags are passed be sure to configure the store
appropriately to ensure that the wasm can actually run instead of
crashing immediately, enabling benchmarking comparisons between these
modes.
* Account for fuel before unconditionally trapping Wasm accesses
Fixes#5445
* Add a test for fuel accounting and unconditionally trapping memory accesses
* cranelift-wasm: translate Wasm loads into lower-level CLIF operations
Rather than using `heap_{load,store,addr}`.
* cranelift: Remove the `heap_{addr,load,store}` instructions
These are now legalized in the `cranelift-wasm` frontend.
* cranelift: Remove the `ir::Heap` entity from CLIF
* Port basic memory operation tests to .wat filetests
* Remove test for verifying CLIF heaps
* Remove `heap_addr` from replace_branching_instructions_and_cfg_predecessors.clif test
* Remove `heap_addr` from readonly.clif test
* Remove `heap_addr` from `table_addr.clif` test
* Remove `heap_addr` from the simd-fvpromote_low.clif test
* Remove `heap_addr` from simd-fvdemote.clif test
* Remove `heap_addr` from the load-op-store.clif test
* Remove the CLIF heap runtest
* Remove `heap_addr` from the global_value.clif test
* Remove `heap_addr` from fpromote.clif runtests
* Remove `heap_addr` from fdemote.clif runtests
* Remove `heap_addr` from memory.clif parser test
* Remove `heap_addr` from reject_load_readonly.clif test
* Remove `heap_addr` from reject_load_notrap.clif test
* Remove `heap_addr` from load_readonly_notrap.clif test
* Remove `static-heap-without-guard-pages.clif` test
Will be subsumed when we port `make-heap-load-store-tests.sh` to generating
`.wat` tests.
* Remove `static-heap-with-guard-pages.clif` test
Will be subsumed when we port `make-heap-load-store-tests.sh` over to `.wat`
tests.
* Remove more heap tests
These will be subsumed by porting `make-heap-load-store-tests.sh` over to `.wat`
tests.
* Remove `heap_addr` from `simple-alias.clif` test
* Remove `heap_addr` from partial-redundancy.clif test
* Remove `heap_addr` from multiple-blocks.clif test
* Remove `heap_addr` from fence.clif test
* Remove `heap_addr` from extends.clif test
* Remove runtests that rely on heaps
Heaps are not a thing in CLIF or the interpreter anymore
* Add generated load/store `.wat` tests
* Enable memory-related wasm features in `.wat` tests
* Remove CLIF heap from fcmp-mem-bug.clif test
* Add a mode for compiling `.wat` all the way to assembly in filetests
* Also generate WAT to assembly tests in `make-load-store-tests.sh`
* cargo fmt
* Reinstate `f{de,pro}mote.clif` tests without the heap bits
* Remove undefined doc link
* Remove outdated SVG and dot file from docs
* Add docs about `None` returns for base address computation helpers
* Factor out `env.heap_access_spectre_mitigation()` to a local
* Expand docs for `FuncEnvironment::heaps` trait method
* Restore f{de,pro}mote+load clif runtests with stack memory
* wip
* start trying to write a runtime test
* cut out all the more complex test cases until i get this one working
* add macro parsing for the trappable error type config
* runtime result tests works for an empty and a string error type
* debugging: macro is broken because interfaces dont have names???
* thats how you name interfaces
* record error and variant error work
* show a concrete trap type, remove debug
* delete clap annotations from wit-bindgen crate
these are not used - clap isnt even an optional dep here - but were a holdover from the old home
Previously, all Wiggle-generated traits were generated with `&mut self`
signatures. With the addition of the `mutable` configuration option to
`from_witx!` and `wasmtime_integration!`, one can disable this, emitting
instead traits that use `&self` (i.e., `mutable: false`). This change is
helpful for implementing wasi-threads: WASI implementations with
interior mutability will now be able to communitcate this to their
Wiggle-generated code.
The other side of this change is the `get_cx` closure passed to Wiggle's
generated `add_to_linker` function. When `mutability` is set to `true`
(default), the `get_cx` function takes a `&mut` data structure from the
store and returns a corresponding `&mut` reference, usually to a field
of the passed-in structure. When `mutability: false`, the `get_cx`
closure will still take a `&mut` data structure but now will return a
`&` reference.
This commit fixes configuration of the pooling instance allocator when
traps are disallowed while fuzzing to ensure that there's at least one
page of memory. The support in `wasm-smith` currently forcibly creates a
1-page memory which was previously invalid in the pooling instance
allocator under the generated configuration, so this commit updates the
configuration generated to ensure that it can fit the generated module.
A mistake was made in the publication of `wit-parser` where a breaking
change was made without bumping its major version, causing build issues
on `main` if `wit-parser` is updated. This commit updates `wit-parser`
to the latest and we'll handle breaking changes better next time.
Closes#5390
This PR reverts #5128 (commit b3333bf9ea),
adding back the ability for the fuzzing config generator to set
the `use_egraphs` Cranelift option. This will start to fuzz the
egraphs-based optimization framework again, now that #5382 has landed.
This commit fixes a bug with components by changing how DWARF
information from a wasm binary is copied over to the final compiled
artifact. Note that this is not the Wasmtime-generated DWARF but rather
the native wasm DWARF itself used in backtraces.
Previously the wasm dwarf was inserted into sections `.*.wasm` where `*`
was `debug_info`, `debug_str`, etc -- one per `gimli::SectionId` as
found in the original wasm module. This does not work with components,
however, where modules did not correctly separate their debug
information into separate sections or otherwise disambiguate. The fix in
this commit is to instead smash all the debug information together into
one large section and store offsets into that giant section. This is
similar to the `name`-section scraping or the trap metadata section
where one section contains all the data for all the modules in a component.
This simplifies the object file parsing by only looking for one section
name and doesn't add all that much complexity to serializing and looking
up dwarf information as well.
* Import Wasmtime support from the `wit-bindgen` repo
This commit imports the `wit-bindgen-gen-host-wasmtime-rust` crate from
the `wit-bindgen` repository into the upstream Wasmtime repository. I've
chosen to not import the full history here since the crate is relatively
small and doesn't have a ton of complexity. While the history of the
crate is quite long the current iteration of the crate's history is
relatively short so there's not a ton of import there anyway. The
thinking is that this can now continue to evolve in-tree.
* Refactor `wasmtime-component-macro` a bit
Make room for a `wit_bindgen` macro to slot in.
* Add initial support for a `bindgen` macro
* Add tests for `wasmtime::component::bindgen!`
* Improve error forgetting `async` feature
* Add end-to-end tests for bindgen
* Add an audit of `unicase`
* Add a license to the test-helpers crate
* Add vet entry for `pulldown-cmark`
* Update publish script with new crate
* Try to fix publish script
* Update audits
* Update lock file
Fix the `instantiate-many` fuzzer from a recent regression introduced
in #5347 where an error message changed slightly. Ideally a concrete
error type would be tested for here but that's deferred to a future PR.
* Add release notes for 3.0.1
* Update some version directives for crates in Wasmtime
* Mark anything with `publish = false` as version 0.0.0
* Mark the icache coherence crate with the same version as Wasmtime
* Fix manifest directives
* Simplify the `ModuleRuntimeInfo` trait slightly
Fold two functions into one as they're only called from one location
anyway.
* Remove ModuleRuntimeInfo::signature
This is redundant as the array mapping is already stored within the
`VMContext` so that can be consulted rather than having a separate trait
function for it. This required altering the `Global` creation slightly
to work correctly in this situation.
* Remove a now-dead constant
* Shared `VMOffsets` across instances
This commit removes the computation of `VMOffsets` to being per-module
instead of per-instance. The `VMOffsets` structure is also quite large
so this shaves off 112 bytes per instance which isn't a huge impact but
should help lower the cost of instantiating small modules.
* Remove `InstanceAllocator::adjust_tunables`
This is no longer needed or necessary with the pooling allocator.
* Fix compile warning
* Fix a vtune warning
* Fix pooling tests
* Fix another test warning
* Add a `WasmBacktrace::new()` constructor
This commit adds a method of manually capturing a backtrace of
WebAssembly frames within a `Store`. The new constructor can be called
with any `AsContext` values, primarily `&Store` and `&Caller`, during
host functions to inspect the calling state.
For now this does not respect the `Config::wasm_backtrace` option and
instead unconditionally captures the backtrace. It's hoped that this can
continue to adapt to needs of embedders by making it more configurable
int he future if necessary.
Closes#5339
* Split `new` into `capture` and `force_capture`
* Remove some custom error types in Wasmtime
These types are mostly cumbersome to work with nowadays that `anyhow` is
used everywhere else. This commit removes `InstantiationError` and
`SetupError` in favor of using `anyhow::Error` throughout. This can
eventually culminate in creation of specific errors for embedders to
downcast to but for now this should be general enough.
* Fix Windows build
* Treat `-` as an alias to `/dev/stdin`
This applies to unix targets only,
as Windows does not have an appropriate alternative.
* Add tests for piped modules from stdin
This applies to unix targets only,
as Windows does not have an appropriate alternative.
* Move precompiled module detection into wasmtime
Previously, wasmtime-cli checked the module to be loaded is
precompiled or not, by pre-opening the given file path to
check if the "\x7FELF" header exists.
This commit moves this branch into the `Module::from_trusted_file`,
which is only invoked with `--allow-precompiled` flag on CLI.
The initial motivation of the commit is, feeding a module to wasmtime
from piped inputs, is blocked by the pre-opening of the module.
The `Module::from_trusted_file`, assumes the --allow-precompiled flag
so there is no piped inputs, happily mmap-ing the module to test
if the header exists.
If --allow-precompiled is not supplied, the existing `Module::from_file`
will be used, without the additional header check as the precompiled
modules are intentionally not allowed on piped inputs for security measures.
One caveat of this approach is that the user may be confused if
he or she tries to execute a precompiled module without
--allow-precompiled, as wasmtime shows an 'input bytes aren't valid
utf-8' error, not directly getting what's going wrong.
So this commit includes a hack-ish workaround for this.
Thanks to @jameysharp for suggesting this idea with a detailed guidance.
* Turn off probestack by default in Cranelift
The probestack feature is not implemented for the aarch64 and s390x
backends and currently the on-by-default status requires the aarch64 and
s390x implementations to be a stub. Turning off probestack by default
allows the s390x and aarch64 backends to panic with an error message to
avoid providing a false sense of security. When the probestack option is
implemented for all backends, however, it may be reasonable to
re-enable.
* aarch64: Improve codegen for AMode fallback
Currently the final fallback for finalizing an `AMode` will generate
both a constant-loading instruction as well as an `add` instruction to
the base register into the same temporary. This commit improves the
codegen by removing the `add` instruction and folding the final add into
the finalized `AMode`. This changes the `extendop` used but both
registers are 64-bit so shouldn't be affected by the extending
operation.
* aarch64: Implement inline stack probes
This commit implements inline stack probes for the aarch64 backend in
Cranelift. The support here is modeled after the x64 support where
unrolled probes are used up to a particular threshold after which a loop
is generated. The instructions here are similar in spirit to x64 except
that unlike x64 the stack pointer isn't modified during the unrolled
loop to avoid needing to re-adjust it back up at the end of the loop.
* Enable inline probestack for AArch64 and Riscv64
This commit enables inline probestacks for the AArch64 and Riscv64
architectures in the same manner that x86_64 has it enabled now. Some
more testing was additionally added since on Unix platforms we should be
guaranteed that Rust's stack overflow message is now printed too.
* Enable probestack for aarch64 in cranelift-fuzzgen
* Address review comments
* Remove implicit stack overflow traps from x64 backend
This commit removes implicit `StackOverflow` traps inserted by the x64
backend for stack-based operations. This was historically required when
stack overflow was detected with page faults but Wasmtime no longer
requires that since it's not suitable for wasm modules which call host
functions. Additionally no other backend implements this form of
implicit trap-code additions so this is intended to synchronize the
behavior of all the backends.
This fixes a test added prior for aarch64 to properly abort the process
instead of accidentally being caught by Wasmtime.
* Fix a style issue
* wasmtime: enable stack probing for x86_64 targets.
This commit unconditionally enables stack probing for x86_64 targets.
On Windows, stack probing is always required because of the way Windows commits
stack pages (via guard page access).
Fixes#5340.
* Remove SIMD types from test case.
- `Store::add_fuel` documentation said it'd panic when the engine is
not configured to have fuel, but it in fact returns an error[1].
- There was a typo in `Store::add_fuel`'s documentation (either either).
I "fixed" the typo by rewording the section using the same wording
as `Store::fuel_consumed` (_if fuel consumption is not enabled
via..._)
[1]ff5abfd993/crates/wasmtime/src/store.rs (L1397-L1400)
The main change here is that io-lifetimes 1.0 switches to use the I/O safety
feature in the standard library rather than providing its own copy.
This also updates to windows-sys 0.42.0 and rustix 0.36.
* Clear affine slots when dropping a `Module`
This commit implements a resource usage optimization for Wasmtime with
the pooling instance allocator by ensuring that when a `Module` is
dropped its backing virtual memory mappings are all removed. Currently
when a `Module` is dropped it releases a strong reference to its
internal memory image but the memory image may stick around in
individual pooling instance allocator slots. When using the `Random`
allocation strategy, for example, this means that the memory images
could stick around for a long time.
While not a pressing issue this has resource usage implications for
Wasmtime. Namely removing a `Module` does not guarantee the memfd, if in
use for a memory image, is closed and deallocated within the kernel.
Unfortunately simply closing the memfd is not sufficient as well as the
mappings into the address space additionally all need to be removed for
the kernel to release the resources for the memfd. This means that to
release all kernel-level resources for a `Module` all slots which have
the memory image mapped in must have the slot reset.
This problem isn't particularly present when using the `NextAvailable`
allocation strategy since the number of lingering memfds is proportional
to the maximum concurrent size of wasm instances. With the `Random` and
`ReuseAffinity` strategies, however, it's much more prominent because
the number of lingering memfds can reach the total number of slots
available. This can appear as a leak of kernel-level memory which can
cause other system instability.
To fix this issue this commit adds necessary instrumentation to `Drop
for Module` to purge all references to the module in the pooling
instance allocator. All index allocation strategies now maintain
affinity tracking to ensure that regardless of the strategy in use a
module that is dropped will remove all its memory mappings. A new
allocation method was added to the index allocator for allocating an
index without setting affinity and only allocating affine slots. This is
used to iterate over all the affine slots without holding the global
index lock for an unnecessarily long time while mappings are removed.
* Review comments
This gets this package to build for `x86_64`, but does not include any
cache clearing for `aarch64-freebsd`. Which will probably build, but not work.
As far as I can tell membarriers are still in the process of being added.
See: https://reviews.freebsd.org/D32360
Since FreeBSD is mirroring the Linux membarrier syscall we may be able
to reuse the same implementation for both Linux and FreeBSD for AArch64.
This commit refactors the internals of `wasmtime_runtime::SharedMemory`
a bit to expose the necessary functions to invoke from the
`wasmtime::SharedMemory` layer. Notably some items are moved out of the
`RwLock` from prior, such as the type and the `VMMemoryDefinition`.
Additionally the organization around the `atomic_*` methods has been
redone to ensure that the `wasmtime`-layer abstraction has a single
method to call into which everything else uses as well.
From the documentation of `CondVar::wait_timeout`:
> The semantics of this function are equivalent to wait except that the thread
> will be blocked for roughly no longer than `dur`. This method should not be
> used for precise timing due to anomalies such as preemption or platform
> differences that might not cause the maximum amount of time waited to be
> precisely `dur`.
Therefore, go to sleep again, if the thread has not slept long enough.
Signed-off-by: Harald Hoyer <harald@profian.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
