This commit updates the implementation of `ComponentType for ()` to
typecheck both the empty tuple type in addition to the `unit` type in
the component model. This allows the usage of `()` when either of those
types are used. Currently this can work because we don't need to
currently support the answer of "what is the type of this host
function". Instead the only question that needs to be answered at
runtime is "does this host function match this type".
This commit updates the lifting and lowering done by Wasmtime to
validate that alignment is all correct. Previously alignment was ignored
because I wasn't sure how this would all work out.
To be extra safe I haven't actually modified any loads/stores and
they're all still unaligned. If this becomes a performance issue we can
investigate aligned loads and stores but otherwise I believe the
requisite locations have been guarded with traps and I've also added
debug asserts to catch possible future mistakes.
This commit updates lifting for integer types and boolean types to
account for WebAssembly/component-model#35 where extra bits are now
discarded instead of being validated as all zero.
This commit splits the current `ComponentValue` trait into three
separate traits:
* `ComponentType` - contains size/align/typecheck information in
addition to the "lower" representation.
* `Lift` - only contains `lift` and `load`
* `Lower` - only contains `lower` and `store`
When describing the original implementation of host functions to Nick he
immediately pointed out this superior solution to the traits involved
with Wasmtime's support for typed parameters/returns in exported and
imported functions. Instead of having dynamic errors at runtime for
things like "you can't lift a `String`" that's instead a static
compile-time error now.
While I was doing this split I also refactored the `ComponentParams`
trait a bit to have `ComponentType` as a supertrait instead of a subtype
which made its implementations a bit more compact. Additionally its impl
blocks were folded into the existing tuple impl blocks.
* Enable passing host functions to components
This commit implements the ability to pass a host function into a
component. The `wasmtime::component::Linker` type now has a `func_wrap`
method allowing it to take a host function which is exposed internally
to the component and available for lowering.
This is currently mostly a "let's get at least the bare minimum working"
implementation. That involves plumbing around lots of various bits of
the canonical ABI and getting all the previous PRs to line up in this
one to get a test where we call a function where the host takes a
string. This PR also additionally starts reading and using the
`may_{enter,leave}` flags since this is the first time they're actually
relevant.
Overall while this is the bare bones of working this is not a final spot
we should end up at. One of the major downsides is that host functions
are represented as:
F: Fn(StoreContextMut<'_, T>, Arg1, Arg2, ...) -> Result<Return>
while this naively seems reasonable this critically doesn't allow
`Return` to actually close over any of its arguments. This means that if
you want to return a string to wasm then it has to be `String` or
`Rc<str>` or some other owned type. In the case of `String` this means
that to return a string to wasm you first have to copy it from the host
to a temporary `String` allocation, then to wasm. This extra copy for
all strings/lists is expected to be prohibitive. Unfortuantely I don't
think Rust is able to solve this, at least on stable, today.
Nevertheless I wanted to at least post this to get some feedback on it
since it's the final step in implementing host imports to see how others
feel about it.
* Fix a typo in an assertion
* Fix some typos
* Review comments
* Add a `VMComponentContext` type and create it on instantiation
This commit fills out the `wasmtime-runtime` crate's support for
`VMComponentContext` and creates it as part of the instantiation
process. This moves a few maps that were temporarily allocated in an
`InstanceData` into the `VMComponentContext` and additionally reads the
canonical options data from there instead.
This type still won't be used in its "full glory" until the lowering of
host functions is completely implemented, however, which will be coming
in a future commit.
* Remove `DerefMut` implementation
* Rebase conflicts
When lifting and lowering for component host imports there won't be a
`Func` available to represent the options and such for the lowering.
That means that the current construction of the `ComponentValue` trait
won't be sufficient for host imports. This commit instead refactors the
traits to instead work with an `Options` type where the `Options` type
can be manufactured from thin air out of the arguments passed to the
component host trampolines.
This new `Options` type is also suitable for storing in `WasmStr` and
`WasmList` to continue to be used to refer back to memory after
these lifted values have been given back to the embedder.
Overall this should largely just be shuffling code around and renaming
`func: &Func` to `options: &Options`.
* Add trampoline compilation support for lowered imports
This commit adds support to the component model implementation for
compiling trampolines suitable for calling host imports. Currently this
is purely just the compilation side of things, modifying the
wasmtime-cranelift crate and additionally filling out a new
`VMComponentOffsets` type (similar to `VMOffsets`). The actual creation
of a `VMComponentContext` is still not performed and will be a
subsequent PR.
Internally though some tests are actually possible with this where we at
least assert that compilation of a component and creation of everything
in-memory doesn't panic or trip any assertions, so some tests are added
here for that as well.
* Fix some test errors
* Implement module imports into components
As a step towards implementing function imports into a component this
commit implements importing modules into a component. This fills out
missing pieces of functionality such as exporting modules as well. The
previous translation code had initial support for translating imported
modules but some of the AST type information was restructured with
feedback from this implementation, namely splitting the
`InstantiateModule` initializer into separate upvar/import variants to
clarify that the item orderings for imports are resolved differently at
runtime.
Much of this commit is also adding infrastructure for any imports at all
into a component. For example a `Linker` type (analagous to
`wasmtime::Linker`) was added here as well. For now this type is quite
limited due to the inability to define host functions (it can only work
with instances and instances-of-modules) but it's enough to start
writing `*.wast` tests which exercise lots of module-related functionality.
* Fix a warning
This commit enhances the processing of components to track all the
dataflow for the processing of `canon.lower`'d functions. At the same
time this fills out a few other missing details to component processing
such as aliasing from some kinds of component instances and similar.
The major changes contained within this are the updates the `info`
submodule which has the AST of component type information. This has been
significantly refactored to prepare for representing lowered functions
and implementing those. The major change is from an `Instantiation` list
to an `Initializer` list which abstractly represents a few other
initialization actions.
This work is split off from my main work to implement component imports
of host functions. This is incomplete in the sense that it doesn't
actually finish everything necessary to define host functions and import
them into components. Instead this is only the changes necessary at the
translation layer (so far). Consequently this commit does not have tests
and also namely doesn't actually include the `VMComponentContext`
initialization and usage. The full body of work is still a bit too messy
to PR just yet so I'm hoping that this is a slimmed-down-enough piece to
adequately be reviewed.
Prior to this PR a major feature of calling component exports (#4039)
was the usage of the `Value<T>` type. This type represents a value
stored in wasm linear memory (the type `T` stored there). This
implementation had a number of drawbacks though:
* When returning a value it's ABI-specific whether you use `T` or
`Value<T>` as a return value. If `T` is represented with one wasm
primitive then you have to return `T`, otherwise the return value must
be `Value<T>`. This is somewhat non-obvious and leaks ABI-details into
the API which is unfortunate.
* The `T` in `Value<T>` was somewhat non-obvious. For example a
wasm-owned string was `Value<String>`. Using `Value<&str>` didn't
work.
* Working with `Value<T>` was unergonomic in the sense that you had to
first "pair" it with a `&Store<U>` to get a `Cursor<T>` and then you
could start reading the value.
* Custom structs and enums, while not implemented yet, were planned to
be quite wonky where when you had `Cursor<MyStruct>` then you would
have to import a `CursorMyStructExt` trait generated by a proc-macro
(think a `#[derive]` on the definition of `MyStruct`) which would
enable field accessors, returning cursors of all the fields.
* In general there was no "generic way" to load a `T` from memory. Other
operations like lift/lower/store all had methods in the
`ComponentValue` trait but load had no equivalent.
None of these drawbacks were deal-breakers per-se. When I started
to implement imported functions, though, the `Value<T>` type no longer
worked. The major difference between imports and exports is that when
receiving values from wasm an export returns at most one wasm primitive
where an import can yield (through arguments) up to 16 wasm primitives.
This means that if an export returned a string it would always be
`Value<String>` but if an import took a string as an argument there was
actually no way to represent this with `Value<String>` since the value
wasn't actually stored in memory but rather the pointer/length pair is
received as arguments. Overall this meant that `Value<T>` couldn't be
used for arguments-to-imports, which means that altogether something new
would be required.
This PR completely removes the `Value<T>` and `Cursor<T>` type in favor
of a different implementation. The inspiration from this comes from the
fact that all primitives can be both lifted and lowered into wasm while
it's just some times which can only go one direction. For example
`String` can be lowered into wasm but can't be lifted from wasm. Instead
some sort of "view" into wasm needs to be created during lifting.
One of the realizations from #4039 was that we could leverage
run-time-type-checking to reject static constructions that don't make
sense. For example if an embedder asserts that a wasm function returns a
Rust `String` we can reject that at typechecking time because it's
impossible for a wasm module to ever do that.
The new system of imports/exports in this PR now looks like:
* Type-checking takes into accont an `Op` operation which indicates
whether we'll be lifting or lowering the type. This means that we can
allow the lowering operation for `String` but disallow the lifting
operation. While we can't statically rule out an embedder saying that
a component returns a `String` we can now reject it at runtime and
disallow it from being called.
* The `ComponentValue` trait now sports a new `load` function. This
function will load and instance of `Self` from the byte-array
provided. This is implemented for all types but only ever actually
executed when the `lift` operation is allowed during type-checking.
* The `Lift` associated type is removed since it's now expected that the
lift operation returns `Self`.
* The `ComponentReturn` trait is now no longer necessary and is removed.
Instead returns are bounded by `ComponentValue`. During type-checking
it's required that the return value can be lifted, disallowing, for
example, returning a `String` or `&str`.
* With `Value` gone there's no need to specify the ABI details of the
return value, or whether it's communicated through memory or not. This
means that handling return values through memory is transparently
handled by Wasmtime.
* Validation is in a sense more eagerly performed now. Whenever a value
`T` is loaded the entire immediate structure of `T` is loaded and
validated. Note that recursive through memory validation still does
not happen, so the contents of lists or strings aren't validated, it's
just validated that the pointers are in-bounds.
Overall this felt like a much clearer system to work with and should be
much easier to integrate with imported functions as well. The new
`WasmStr` and `WasmList<T>` types can be used in import arguments and
lifted from the immediate arguments provided rather than forcing them to
always be stored in memory.
* Add a first-class `StoreId` type to Wasmtime
This commit adds a `StoreId` type to uniquely identify a store
internally within Wasmtime. This hasn't been created previously as it
was never really needed but I've run across a case for its usage in the
component model so I've gone ahead and split out a commit to add this type.
While I was here in this file I opted to improve some other
miscellaneous things as well:
* Notes were added to the `Index` impls that unchecked indexing could be
used in theory if we ever need it one day.
* The check in `Index` for the same store should now be a bit lighter on
codegen where instead of having a `panic!()` in the codegen for each
`Index` there's now an out-of-line version which is `#[cold]`. This
should improve codegen as calling a function with no arguments is
slighly more efficient than calling the panic macro with one string argument.
* An `assert!` guarded with a `cfg(debug_assertions)` was changed to a
`debug_assert!`.
* Allocation of a `StoreId` was refactored to a method on the `StoreId`
itself.
* Review comments
* Fix an ordering
* Change some `VMContext` pointers to `()` pointers
This commit is motivated by my work on the component model
implementation for imported functions. Currently all context pointers in
wasm are `*mut VMContext` but with the component model my plan is to
make some pointers instead along the lines of `*mut VMComponentContext`.
In doing this though one worry I have is breaking what has otherwise
been a core invariant of Wasmtime for quite some time, subtly
introducing bugs by accident.
To help assuage my worry I've opted here to erase knowledge of
`*mut VMContext` where possible. Instead where applicable a context
pointer is simply known as `*mut ()` and the embedder doesn't actually
know anything about this context beyond the value of the pointer. This
will help prevent Wasmtime from accidentally ever trying to interpret
this context pointer as an actual `VMContext` when it might instead be a
`VMComponentContext`.
Overall this was a pretty smooth transition. The main change here is
that the `VMTrampoline` (now sporting more docs) has its first argument
changed to `*mut ()`. The second argument, the caller context, is still
configured as `*mut VMContext` though because all functions are always
called from wasm still. Eventually for component-to-component calls I
think we'll probably "fake" the second argument as the same as the first
argument, losing track of the original caller, as an intentional way of
isolating components from each other.
Along the way there are a few host locations which do actually assume
that the first argument is indeed a `VMContext`. These are valid
assumptions that are upheld from a correct implementation, but I opted
to add a "magic" field to `VMContext` to assert this in debug mode. This
new "magic" field is inintialized during normal vmcontext initialization
and it's checked whenever a `VMContext` is reinterpreted as an
`Instance` (but only in debug mode). My hope here is to catch any future
accidental mistakes, if ever.
* Use a VMOpaqueContext wrapper
* Fix typos
* Change wasm-to-host trampolines to take the values_vec size
This commit changes the ABI of wasm-to-host trampolines, which are
only used right now for functions created with `Func::new`, to pass
along the size of the `values_vec` argument. Previously the trampoline
simply received `*mut ValRaw` and assumed that it was the appropriate
size. By receiving a size as well we can thread through `&mut [ValRaw]`
internally instead of `*mut ValRaw`.
The original motivation for this is that I'm planning to leverage these
trampolines for the component model for host-defined functions. Out of
an abundance of caution of making sure that everything lines up I wanted
to be able to write down asserts about the size received at runtime
compared to the size expected. This overall led me to the desire to
thread this size parameter through on the assumption that it would not
impact performance all that much.
I ran two benchmarks locally from the `call.rs` benchmark and got:
* `sync/no-hook/wasm-to-host - nop - unchecked` - no change
* `sync/no-hook/wasm-to-host - nop-params-and-results - unchecked` - 5%
slower
This is what I roughly expected in that if nothing actually reads the
new parameter (e.g. no arguments) then threading through the parameter
is effectively otherwise free. Otherwise though accesses to the `ValRaw`
storage is now bounds-checked internally in Wasmtime instead of assuming
it's valid, leading to the 5% slowdown (~9.6ns to ~10.3ns). If this
becomes a peformance bottleneck for a particular use case then we should
be fine to remove the bounds checking here or otherwise only bounds
check in debug mode, otherwise I plan on leaving this as-is.
Of particular note this also changes the C API for `*_unchecked`
functions where the C callback now receives the size of the array as
well.
* Add docs
The `wasmtime-cpp` test suite uncovered an issue where asking for the
frames of a trap would fail immediately after the trap was created. In
addition to fixing this issue I've also updated the documentation of
`Trap::frames` to indicate when it returns `None`.
* sorta working in runtime
* wasmtime-runtime: get rid of wasm-backtrace feature
* wasmtime: factor to make backtraces recording optional. not configurable yet
* get rid of wasm-backtrace features
* trap tests: now a Trap optionally contains backtrace
* eliminate wasm-backtrace feature
* code review fixes
* ci: no more wasm-backtrace feature
* c_api: backtraces always enabled
* config: unwind required by backtraces and ref types
* plumbed
* test that disabling backtraces works
* code review comments
* fuzzing generator: wasm_backtrace is a runtime config now
* doc fix
* Make `ValRaw` fields private
Force accessing to go through constructors and accessors to localize the
knowledge about little-endian-ness. This is spawned since I made a
mistake in #4039 about endianness.
* Fix some tests
* Component model changes
* components: Implement the ability to call component exports
This commit is an implementation of the typed method of calling
component exports. This is intended to represent the most efficient way
of calling a component in Wasmtime, similar to what `TypedFunc`
represents today for core wasm.
Internally this contains all the traits and implementations necessary to
invoke component exports with any type signature (e.g. arbitrary
parameters and/or results). The expectation is that for results we'll
reuse all of this infrastructure except in reverse (arguments and
results will be swapped when defining imports).
Some features of this implementation are:
* Arbitrary type hierarchies are supported
* The Rust-standard `Option`, `Result`, `String`, `Vec<T>`, and tuple
types all map down to the corresponding type in the component model.
* Basic utf-16 string support is implemented as proof-of-concept to show
what handling might look like. This will need further testing and
benchmarking.
* Arguments can be behind "smart pointers", so for example
`&Rc<Arc<[u8]>>` corresponds to `list<u8>` in interface types.
* Bulk copies from linear memory never happen unless explicitly
instructed to do so.
The goal of this commit is to create the ability to actually invoke wasm
components. This represents what is expected to be the performance
threshold for these calls where it ideally should be optimal how
WebAssembly is invoked. One major missing piece of this is a `#[derive]`
of some sort to generate Rust types for arbitrary `*.wit` types such as
custom records, variants, flags, unions, etc. The current trait impls
for tuples and `Result<T, E>` are expected to have fleshed out most of
what such a derive would look like.
There are some downsides and missing pieces to this commit and method of
calling components, however, such as:
* Passing `&[u8]` to WebAssembly is currently not optimal. Ideally this
compiles down to a `memcpy`-equivalent somewhere but that currently
doesn't happen due to all the bounds checks of copying data into
memory. I have been unsuccessful so far at getting these bounds checks
to be removed.
* There is no finalization at this time (the "post return" functionality
in the canonical ABI). Implementing this should be relatively
straightforward but at this time requires `wasmparser` changes to
catch up with the current canonical ABI.
* There is no guarantee that results of a wasm function will be
validated. As results are consumed they are validated but this means
that if function returns an invalid string which the host doesn't look
at then no trap will be generated. This is probably not the intended
semantics of hosts in the component model.
* At this time there's no support for memory64 memories, just a bunch of
`FIXME`s to get around to. It's expected that this won't be too
onerous, however. Some extra care will need to ensure that the various
methods related to size/alignment all optimize to the same thing they
do today (e.g. constants).
* The return value of a typed component function is either `T` or
`Value<T>`, and it depends on the ABI details of `T` and whether it
takes up more than one return value slot or not. This is an
ABI-implementation detail which is being forced through to the API
layer which is pretty unfortunate. For example if you say the return
value of a function is `(u8, u32)` then it's a runtime type-checking
error. I don't know of a great way to solve this at this time.
Overall I'm feeling optimistic about this trajectory of implementing
value lifting/lowering in Wasmtime. While there are a number of
downsides none seem completely insurmountable. There's naturally still a
good deal of work with the component model but this should be a
significant step up towards implementing and testing the component model.
* Review comments
* Write tests for calling functions
This commit adds a new test file for actually executing functions and
testing their results. This is not written as a `*.wast` test yet since
it's not 100% clear if that's the best way to do that for now (given
that dynamic signatures aren't supported yet). The tests themselves
could all largely be translated to `*.wast` testing in the future,
though, if supported.
Along the way a number of minor issues were fixed with lowerings with
the bugs exposed here.
* Fix an endian mistake
* Fix a typo and the `memory.fill` instruction
* Initial skeleton of some component model processing
This commit is the first of what will likely be many to implement the
component model proposal in Wasmtime. This will be structured as a
series of incremental commits, most of which haven't been written yet.
My hope is to make this incremental and over time to make this easier to
review and easier to test each step in isolation.
Here much of the skeleton of how components are going to work in
Wasmtime is sketched out. This is not a complete implementation of the
component model so it's not all that useful yet, but some things you can
do are:
* Process the type section into a representation amenable for working
with in Wasmtime.
* Process the module section and register core wasm modules.
* Process the instance section for core wasm modules.
* Process core wasm module imports.
* Process core wasm instance aliasing.
* Ability to compile a component with core wasm embedded.
* Ability to instantiate a component with no imports.
* Ability to get functions from this component.
This is already starting to diverge from the previous module linking
representation where a `Component` will try to avoid unnecessary
metadata about the component and instead internally only have the bare
minimum necessary to instantiate the module. My hope is we can avoid
constructing most of the index spaces during instantiation only for it
to all ge thrown away. Additionally I'm predicting that we'll need to
see through processing where possible to know how to generate adapters
and where they are fused.
At this time you can't actually call a component's functions, and that's
the next PR that I would like to make.
* Add tests for the component model support
This commit uses the recently updated wasm-tools crates to add tests for
the component model added in the previous commit. This involved updating
the `wasmtime-wast` crate for component-model changes. Currently the
component support there is quite primitive, but enough to at least
instantiate components and verify the internals of Wasmtime are all
working correctly. Additionally some simple tests for the embedding API
have also been added.
* Improve the `wasmtime` crate's README
This commit is me finally getting back to #2688 and improving the README
of the `wasmtime` crate. Currently we have a [pretty drab README][drab]
that doesn't really convey what we want about Wasmtime.
While I was doing this I opted to update the feature list of Wasmtime as
well in the main README (which is mirrored into the crate readme),
namely adding a bullet point for "secure" which I felt was missing
relative to how we think about Wasmtime.
Naturally there's a lot of ways to paint this shed, so feedback is of
course welcome on this! (I'm not the best writer myself)
[drab]: https://crates.io/crates/wasmtime/0.37.0
* Expand the "Fast" bullet a bit more
* Reference the book from the wasmtime crate
* Update more security docs
Also merge the sandboxing security page with the main security page to
avoid the empty security page.
* Improve documentation around `ResourceLimiter`
This commit takes a pass through the `Store::limiter` method and related
types/traits to improve the documentation with an example and soup up
any recent developments in the documentation.
Closes#4138
* Fix a broken doc link
* Update the wasm-tools family of crates
This commit updates these crates as used by Wasmtime for the recently
published versions to pull in changes necessary to support the component
model. I've split this out from #4005 to make it clear what's impacted
here and #4005 can simply rebase on top of this to pick up the necessary
changes.
* More test fixes
* Run a callback when the interruption epoch is reached
Adds Store::epoch_deadline_callback. This accepts a callback which, when
invoked, can mutate the store's contents. The callback can either return
an error (in which case we trap) or return a delta which we'll use to
set the new epoch deadline.
* Add a basic test for epoch interruption callback
* Some small nits
- Remove use of &mut in the pattern match
- Return both yields and state from run_and_count_yields_or_trap in
test code and assert on them separately.
- Add a test for trapping on a state failure.
Currently, we have partial Spectre mitigation: we protect heap accesses
with dynamic bounds checks. Specifically, we guard against errant
accesses on the misspeculated path beyond the bounds-check conditional
branch by adding a conditional move that is also dependent on the
bounds-check condition. This data dependency on the condition is not
speculated and thus will always pick the "safe" value (in the heap case,
a NULL address) on the misspeculated path, until the pipeline flushes
and recovers onto the correct path.
This PR uses the same technique both for table accesses -- used to
implement Wasm tables -- and for jumptables, used to implement Wasm
`br_table` instructions.
In the case of Wasm tables, the cmove picks the table base address on
the misspeculated path. This is equivalent to reading the first table
entry. This prevents loads of arbitrary data addresses on the
misspeculated path.
In the case of `br_table`, the cmove picks index 0 on the misspeculated
path. This is safer than allowing a branch to an address loaded from an
index under misspeculation (i.e., it preserves control-flow integrity
even under misspeculation).
The table mitigation is controlled by a Cranelift setting, on by
default. The br_table mitigation is always on, because it is part of the
single lowering pseudoinstruction. In both cases, the impact should be
minimal: a single extra cmove in a (relatively) rarely-used operation.
The table mitigation is architecture-independent (happens during
legalization); the br_table mitigation has been implemented for both x64
and aarch64. (I don't know enough about s390x to implement this
confidently there, but would happily review a PR to do the same on that
platform.)
While working with the runtime `Memory` object, it became clear that
some refactoring was needed. In order to implement shared memory from
the threads proposal, we must be able to atomically change the memory
size. Previously, the split into variants, `Memory::Static` and
`Memory::Dynamic`, made any attempt to lock forced us to duplicate logic
in various places.
This change moves `enum Memory { Static..., Dynamic... }` to simply
`struct Memory(Box<dyn RuntimeLinearMemory>)`. A new type,
`ExternalMemory`, takes the place of `Memory::Static` and also
implements the `RuntimeLinearMemory` trait, allowing `Memory` to contain
the same two options as before: `MmapMemory` for `Memory::Dynamic` and
`ExternalMemory` for `Memory::Static`. To interface with the
`PoolingAllocator`, this change also required the ability to downcast to
the internal representation.
* Run a `cargo update` over our dependencies
This'll notably fix a `cargo audit` error where we have a pinned version
of the `regex` crate which has a CVE assigned to it.
* Update to `object` and `hashbrown` crates
Prune some duplicate versions showing up from the previous `cargo update`
* Reduce contention on the global module rwlock
This commit intendes to close#4025 by reducing contention on the global
rwlock Wasmtime has for module information during instantiation and
dropping a store. Currently registration of a module into this global
map happens during instantiation, but this can be a hot path as
embeddings may want to, in parallel, instantiate modules.
Instead this switches to a strategy of inserting into the global module
map when a `Module` is created and then removing it from the map when
the `Module` is dropped. Registration in a `Store` now preserves the
entire `Module` within the store as opposed to trying to only save it
piecemeal. In reality the only piece that wasn't saved within a store
was the `TypeTables` which was pretty inconsequential for core wasm
modules anyway.
This means that instantiation should now clone a singluar `Arc` into a
`Store` per `Module` (previously it cloned two) with zero managemnt on
the global rwlock as that happened at `Module` creation time.
Additionally dropping a `Store` again involves zero rwlock management
and only a single `Arc` drop per-instantiated module (previously it was
two).
In the process of doing this I also went ahead and removed the
`Module::new_with_name` API. This has been difficult to support
historically with various variations on the internals of `ModuleInner`
because it involves mutating a `Module` after it's been created. My hope
is that this API is pretty rarely used and/or isn't super important, so
it's ok to remove.
Finally this change removes some internal `Arc` layerings that are no
longer necessary, attempting to use either `T` or `&T` where possible
without dealing with the overhead of an `Arc`.
Closes#4025
* Move back to a `BTreeMap` in `ModuleRegistry`
This commit implements an optimization to help improve concurrently
creating instances of a module on many threads simultaneously. One
bottleneck to this measured has been the reference count modification on
`Arc<HostFunc>`. Each host function stored within a `Linker<T>` is
wrapped in an `Arc<HostFunc>` structure, and when any of those host
functions are inserted into a store the reference count is incremented.
When the store is dropped the reference count is then decremented.
This ends up meaning that when a module imports N functions it ends up
doing 2N atomic modifications over the lifetime of the instance. For
embeddings where the `Linker<T>` is rarely modified but instances are
frequently created this can be a surprising bottleneck to creating many
instances.
A change implemented here is to optimize the instantiation process when
using an `InstancePre<T>`. An `InstancePre` serves as an opportunity to
take the list of items used to instantiate a module and wrap them all up
in an `Arc<[T]>`. Everything is going to get cloned into a `Store<T>`
anyway so to optimize this the `Arc<[T]>` is cloned at the top-level and
then nothing else is cloned internally. This continues to, however,
preserve a strong reference count for all contained items to prevent
them from being deallocated.
A new variant of `FuncKind` was added for host functions which is
effectively stored via `*mut HostFunc`. This variant is unsafe to create
and manage and has been documented internally.
Performance-wise the overall impact of this change is somewhat minor.
It's already a bit esoteric if this atomic increment and decrement are a
bottleneck due to the number of concurrent instances being created. In
my measurements I've seen that this can reduce instantiation time by up
to 10% for a module that imports two dozen functions. For larger modules
with more imports this is expected to have a larger win.
With these fixes, all this PR has to do is instantiate and run the
checker on the `regalloc2::Output`. This is off by default, and is
enabled by setting the `regalloc_checker` Cranelift option.
This restores the old functionality provided by e.g. the
`backtracking_checked` regalloc algorithm setting rather than
`backtracking` when we were still on regalloc.rs.
This commit removes support for the `userfaultfd` or "uffd" syscall on
Linux. This support was originally added for users migrating from Lucet
to Wasmtime, but the recent developments of kernel-supported
copy-on-write support for memory initialization wound up being more
appropriate for these use cases than usefaultfd. The main reason for
moving to copy-on-write initialization are:
* The `userfaultfd` feature was never necessarily intended for this
style of use case with wasm and was susceptible to subtle and rare
bugs that were extremely difficult to track down. We were never 100%
certain that there were kernel bugs related to userfaultfd but the
suspicion never went away.
* Handling faults with userfaultfd was always slow and single-threaded.
Only one thread could handle faults and traveling to user-space to
handle faults is inherently slower than handling them all in the
kernel. The single-threaded aspect in particular presented a
significant scaling bottleneck for embeddings that want to run many
wasm instances in parallel.
* One of the major benefits of userfaultfd was lazy initialization of
wasm linear memory which is also achieved with the copy-on-write
initialization support we have right now.
* One of the suspected benefits of userfaultfd was less frobbing of the
kernel vma structures when wasm modules are instantiated. Currently
the copy-on-write support has a mitigation where we attempt to reuse
the memory images where possible to avoid changing vma structures.
When comparing this to userfaultfd's performance it was found that
kernel modifications of vmas aren't a worrisome bottleneck so
copy-on-write is suitable for this as well.
Overall there are no remaining benefits that userfaultfd gives that
copy-on-write doesn't, and copy-on-write solves a major downsides of
userfaultfd, the scaling issue with a single faulting thread.
Additionally copy-on-write support seems much more robust in terms of
kernel implementation since it's only using standard memory-management
syscalls which are heavily exercised. Finally copy-on-write support
provides a new bonus where read-only memory in WebAssembly can be mapped
directly to the same kernel cache page, even amongst many wasm instances
of the same module, which was never possible with userfaultfd.
In light of all this it's expected that all users of userfaultfd should
migrate to the copy-on-write initialization of Wasmtime (which is
enabled by default).
* Store the `ValRaw` type in little-endian format
This commit changes the internal representation of the `ValRaw` type to
an unconditionally little-endian format instead of its current
native-endian format. The documentation and various accessors here have
been updated as well as the associated trampolines that read `ValRaw`
to always work with little-endian values, converting to the host
endianness as necessary.
The motivation for this change originally comes from the implementation
of the component model that I'm working on. One aspect of the component
model's canonical ABI is how variants are passed to functions as
immediate arguments. For example for a component model function:
```
foo: function(x: expected<i32, f64>)
```
This translates to a core wasm function:
```wasm
(module
(func (export "foo") (param i32 i64)
;; ...
)
)
```
The first `i32` parameter to the core wasm function is the discriminant
of whether the result is an "ok" or an "err". The second `i64`, however,
is the "join" operation on the `i32` and `f64` payloads. Essentially
these two types are unioned into one type to get passed into the function.
Currently in the implementation of the component model my plan is to
construct a `*mut [ValRaw]` to pass through to WebAssembly, always
invoking component exports through host trampolines. This means that the
implementation for `Result<T, E>` needs to do the correct "join"
operation here when encoding a particular case into the corresponding
`ValRaw`.
I personally found this particularly tricky to do structurally. The
solution that I settled on with fitzgen was that if `ValRaw` was always
stored in a little endian format then we could employ a trick where when
encoding a variant we first set all the `ValRaw` slots to zero, then the
associated case we have is encoding. Afterwards the `ValRaw` values are
already encoded into the correct format as if they'd been "join"ed.
For example if we were to encode `Ok(1i32)` then this would produce
`ValRaw { i32: 1 }`, which memory-wise is equivalent to `ValRaw { i64: 1 }`
if the other bytes in the `ValRaw` are guaranteed to be zero. Similarly
storing `ValRaw { f64 }` is equivalent to the storage required for
`ValRaw { i64 }` here in the join operation.
Note, though, that this equivalence relies on everything being
little-endian. Otherwise the in-memory representations of `ValRaw { i32: 1 }`
and `ValRaw { i64: 1 }` are different.
That motivation is what leads to this change. It's expected that this is
a low-to-zero cost change in the sense that little-endian platforms will
see no change and big-endian platforms are already required to
efficiently byte-swap loads/stores as WebAssembly requires that.
Additionally the `ValRaw` type is an esoteric niche use case primarily
used for accelerating the C API right now, so it's expected that not
many users will have to update for this change.
* Track down some more endianness conversions
This commit adds a method, `Module::initialize_copy_on_write_image`,
which will explicitly create the copy-on-write image during the method
call instead of lazily deferring it until the first instantiation of a
module. Given the relative expense of creation of a memfd on Linux this
can otherwise run the risk of unnaturally perturbing the
time-of-instantiation of a module. Additionally whenever lazy
initialization is provided in an API it's typically good practice to
also have an optionally-called forced initialization.
* Update wasm-tools crates
This commit updates the wasm-tools family of crates as used in Wasmtime.
Notably this brings in the update which removes module linking support
as well as a number of internal refactorings around names and such
within wasmparser itself. This updates all of the wasm translation
support which binds to wasmparser as appropriate.
Other crates all had API-compatible changes for at least what Wasmtime
used so no further changes were necessary beyond updating version
requirements.
* Update a test expectation
* Refactor away the `Instantiator` type in Wasmtime
This internal type in Wasmtime was primarily used for the module linking
proposal to handle instantiation of many instances and refactor out the
sync and async parts to minimize duplication. With the removal of the
module linking proposal, however, this type isn't really necessary any
longer. In working to implement the component model proposal I was
looking already to refactor this and I figured it'd be good to land that
ahead of time on `main` separate of other refactorings.
This commit removes the `Instantiator` type in the `instance` module.
The type was already private to Wasmtime so this shouldn't have any
impact on consumers. This allows simplifying various code paths to avoid
another abstraction. The meat of instantiation is moved to
`Instance::new_raw` which should be reusable for the component model as
well.
One bug is actually fixed in this commit as well where
`Linker::instantiate` and `InstancePre::instantiate` failed to check
that async support was disabled on a store. This means that they could
have led to a panic if used with an async store and a start function
called an async import (or an async resource limiter yielded). A few
tests were updated with this.
* Review comments
* Upgrade all crates to the Rust 2021 edition
I've personally started using the new format strings for things like
`panic!("some message {foo}")` or similar and have been upgrading crates
on a case-by-case basis, but I think it probably makes more sense to go
ahead and blanket upgrade everything so 2021 features are always
available.
* Fix compile of the C API
* Fix a warning
* Fix another warning
* Bump to 0.36.0
* Add a two-week delay to Wasmtime's release process
This commit is a proposal to update Wasmtime's release process with a
two-week delay from branching a release until it's actually officially
released. We've had two issues lately that came up which led to this proposal:
* In #3915 it was realized that changes just before the 0.35.0 release
weren't enough for an embedding use case, but the PR didn't meet the
expectations for a full patch release.
* At Fastly we were about to start rolling out a new version of Wasmtime
when over the weekend the fuzz bug #3951 was found. This led to the
desire internally to have a "must have been fuzzed for this long"
period of time for Wasmtime changes which we felt were better
reflected in the release process itself rather than something about
Fastly's own integration with Wasmtime.
This commit updates the automation for releases to unconditionally
create a `release-X.Y.Z` branch on the 5th of every month. The actual
release from this branch is then performed on the 20th of every month,
roughly two weeks later. This should provide a period of time to ensure
that all changes in a release are fuzzed for at least two weeks and
avoid any further surprises. This should also help with any last-minute
changes made just before a release if they need tweaking since
backporting to a not-yet-released branch is much easier.
Overall there are some new properties about Wasmtime with this proposal
as well:
* The `main` branch will always have a section in `RELEASES.md` which is
listed as "Unreleased" for us to fill out.
* The `main` branch will always be a version ahead of the latest
release. For example it will be bump pre-emptively as part of the
release process on the 5th where if `release-2.0.0` was created then
the `main` branch will have 3.0.0 Wasmtime.
* Dates for major versions are automatically updated in the
`RELEASES.md` notes.
The associated documentation for our release process is updated and the
various scripts should all be updated now as well with this commit.
* Add notes on a security patch
* Clarify security fixes shouldn't be previewed early on CI
This splits the existing `lookup_by_declaration` function into a
lookup-per-type-of-item. This refactor ends up cleaning up a fair bit of
code in the `wasmtime` crate by removing a number of `unreachable!()`
blocks which are now no longer necessary.
* Remove duplicate `TypeTables` type
This was once needed historically but it is no longer needed.
* Make the internals of `TypeTables` private
Instead of reaching internally for the `wasm_signatures` map an `Index`
implementation now exists to indirect accesses through the type of the
index being accessed. For the component model this table of types will
grow a number of other tables and this'll assist in consuming sites not
having to worry so much about which map they're reaching into.
* Remove the module linking implementation in Wasmtime
This commit removes the experimental implementation of the module
linking WebAssembly proposal from Wasmtime. The module linking is no
longer intended for core WebAssembly but is instead incorporated into
the component model now at this point. This means that very large parts
of Wasmtime's implementation of module linking are no longer applicable
and would change greatly with an implementation of the component model.
The main purpose of this is to remove Wasmtime's reliance on the support
for module-linking in `wasmparser` and tooling crates. With this
reliance removed we can move over to the `component-model` branch of
`wasmparser` and use the updated support for the component model.
Additionally given the trajectory of the component model proposal the
embedding API of Wasmtime will not look like what it looks like today
for WebAssembly. For example the core wasm `Instance` will not change
and instead a `Component` is likely to be added instead.
Some more rationale for this is in #3941, but the basic idea is that I
feel that it's not going to be viable to develop support for the
component model on a non-`main` branch of Wasmtime. Additionaly I don't
think it's viable, for the same reasons as `wasm-tools`, to support the
old module linking proposal and the new component model at the same
time.
This commit takes a moment to not only delete the existing module
linking implementation but some abstractions are also simplified. For
example module serialization is a bit simpler that there's only one
module. Additionally instantiation is much simpler since the only
initializer we have to deal with are imports and nothing else.
Closes#3941
* Fix doc link
* Update comments
* Instead of simply panicking, return an error when we attempt to resume on a dying fiber.
This situation should never occur in the existing code base, but can be
triggered if support for running outside async code in a call hook.
* Shift `async_cx()` to return an `Option`, reflecting if the fiber is dying.
This should never happen in the existing code base, but is a nice
forward-looking guard. The current implementations simply lift the
trap that would eventually be produced by such an operation into
a `Trap` (or similar) at the invocation of `async_cx()`.
* Add support for using `async` call hooks.
This retains the ability to do non-async hooks. Hooks end up being
implemented as an async trait with a handler call, to get around some
issues passing around async closures. This change requires some of
the prior changes to handle picking up blocked tasks during fiber
shutdown, to avoid some panics during timeouts and other such events.
* More fully specify a doc link, to avoid a doc-building error.
* Revert the use of catchable traps on cancellation of a fiber; turn them into expect()/unwrap().
The justification for this revert is that (a) these events shouldn't
happen, and (b) they wouldn't be catchable by wasm anyways.
* Replace a duplicated check in `async` hook evaluation with a single check.
This also moves the checks inside of their respective Async variants,
meaning that if you're using an async-enabled version of wasmtime but
using the synchronous versions of the callbacks, you won't pay any
penalty for validating the async context.
* Use `match &mut ...` insead of `ref mut`.
* Add some documentation on why/when `async_cx` can return None.
* Add two simple test cases for async call hooks.
* Fix async_cx() to check both the box and the value for current_poll_cx.
In the prior version, we only checked that the box had not been cleared,
but had not ensured that there was an actual context for us to use. This
updates the check to validate both, returning None if the inner context
is missing. This allows us to skip a validation check inside `block_on`,
since all callers will have run through the `async_cx` check prior to
arrival.
* Tweak the timeout test to address PR suggestions.
* Add a test about dropping async hooks while suspended
Should help exercise that the check for `None` is properly handled in a
few more locations.
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
This commit exposes some various details and config options for having
finer-grain control over mlock-ing the memory of modules. This amounts
to three different changes being present in this commit:
* A new `Module::image_range` API is added to expose the range in host
memory of where the compiled image resides. This enables embedders to
make mlock-ing decisions independently of Wasmtime. Otherwise though
there's not too much useful that can be done with this range
information at this time.
* A new `Config::force_memory_init_memfd` option has been added. This
option is used to force the usage of `memfd_create` on Linux even when
the original module comes from a file on disk. With mlock-ing the main
purpose for Wasmtime is likely to be avoiding major page faults that
go back to disk, so this is another major source of avoiding page
faults by ensuring that the initialization contents of memory are
always in RAM.
* The `memory_images` field of a `Module` has gone back to being lazily
created on the first instantiation, effectively reverting #3914. This
enables embedders to defer the creation of the image to as late as
possible to allow modules to be created from precompiled images
without actually loading all the contents of the data segments from
disk immediately.
These changes are all somewhat low-level controls which aren't intended
to be generally used by embedders. If fine-grained control is desired
though it's hoped that these knobs provide what's necessary to be
achieved.
