0d829a57eef3e66c51177e72ec2e2d692e7173d2
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Alex Crichton
|
3339dd1f01 |
Implement the post-return attribute (#4297)
This commit implements the `post-return` feature of the canonical ABI in the component model. This attribute is an optionally-specified function which is to be executed after the return value has been processed by the caller to optionally clean-up the return value. This enables, for example, returning an allocated string and the host then knows how to clean it up to prevent memory leaks in the original module. The API exposed in this PR changes the prior `TypedFunc::call` API in behavior but not in its signature. Previously the `TypedFunc::call` method would set the `may_enter` flag on the way out, but now that operation is deferred until a new `TypedFunc::post_return` method is called. This means that once a method on an instance is invoked then nothing else can be done on the instance until the `post_return` method is called. Note that the method must be called irrespective of whether the `post-return` canonical ABI option was specified or not. Internally wasm will be invoked if necessary. This is a pretty wonky and unergonomic API to work with. For now I couldn't think of a better alternative that improved on the ergonomics. In the theory that the raw Wasmtime bindings for a component may not be used all that heavily (instead `wit-bindgen` would largely be used) I'm hoping that this isn't too much of an issue in the future. cc #4185 |
||
|
Alex Crichton
|
651f40855f |
Add support for nested components (#4285)
* Add support for nested components
This commit is an implementation of a number of features of the
component model including:
* Defining nested components
* Outer aliases to components and modules
* Instantiating nested components
The implementation here is intended to be a foundational pillar of
Wasmtime's component model support since recursion and nested components
are the bread-and-butter of the component model. At a high level the
intention for the component model implementation in Wasmtime has long
been that the recursive nature of components is "erased" at compile time
to something that's more optimized and efficient to process. This commit
ended up exemplifying this quite well where the vast majority of the
internal changes here are in the "compilation" phase of a component
rather than the runtime instantiation phase. The support in the
`wasmtime` crate, the runtime instantiation support, only had minor
updates here while the internals of translation have seen heavy updates.
The `translate` module was greatly refactored here in this commit.
Previously it would, as a component is parsed, create a final
`Component` to hand off to trampoline compilation and get persisted at
runtime. Instead now it's a thin layer over `wasmparser` which simply
records a list of `LocalInitializer` entries for how to instantiate the
component and its index spaces are built. This internal representation
of the instantiation of a component is pretty close to the binary format
intentionally.
Instead of performing dataflow legwork the `translate` phase of a
component is now responsible for two primary tasks:
1. All components and modules are discovered within a component. They're
assigned `Static{Component,Module}Index` depending on where they're
found and a `{Module,}Translation` is prepared for each one. This
"flattens" the recursive structure of the binary into an indexed list
processable later.
2. The lexical scope of components is managed here to implement outer
module and component aliases. This is a significant design
implementation because when closing over an outer component or module
that item may actually be imported or something like the result of a
previous instantiation. This means that the capture of
modules and components is both a lexical concern as well as a runtime
concern. The handling of the "runtime" bits are handled in the next
phase of compilation.
The next and currently final phase of compilation is a new pass where
much of the historical code in `translate.rs` has been moved to (but
heavily refactored). The goal of compilation is to produce one "flat"
list of initializers for a component (as happens prior to this PR) and
to achieve this an "inliner" phase runs which runs through the
instantiation process at compile time to produce a list of initializers.
This `inline` module is the main addition as part of this PR and is now
the workhorse for dataflow analysis and tracking what's actually
referring to what.
During the `inline` phase the local initializers recorded in the
`translate` phase are processed, in sequence, to instantiate a
component. Definitions of items are tracked to correspond to their root
definition which allows seeing across instantiation argument boundaries
and such. Handling "upvars" for component outer aliases is handled in
the `inline` phase as well by creating state for a component whenever a
component is defined as was recorded during the `translate` phase.
Finally this phase is chiefly responsible for doing all string-based
name resolution at compile time that it can. This means that at runtime
no string maps will need to be consulted for item exports and such.
The final result of inlining is a list of "global initializers" which is
a flat list processed during instantiation time. These are almost
identical to the initializers that were processed prior to this PR.
There are certainly still more gaps of the component model to implement
but this should be a major leg up in terms of functionality that
Wasmtime implements. This commit, however leaves behind a "hole" which
is not intended to be filled in at this time, namely importing and
exporting components at the "root" level from and to the host. This is
tracked and explained in more detail as part of #4283.
cc #4185 as this completes a number of items there
* Tweak code to work on stable without warning
* Review comments
|
||
|
Alex Crichton
|
7d7ddceb17 |
Update wasm-tools crates (#4246)
This commit updates the wasm-tools family of crates, notably pulling in the refactorings and updates from bytecodealliance/wasm-tools#621 for the latest iteration of the component model. This commit additionally updates all support for the component model for these changes, notably: * Many bits and pieces of type information was refactored. Many `FooTypeIndex` namings are now `TypeFooIndex`. Additionally there is now `TypeIndex` as well as `ComponentTypeIndex` for the two type index spaces in a component. * A number of new sections are now processed to handle the core and component variants. * Internal maps were split such as the `funcs` map into `component_funcs` and `funcs` (same for `instances`). * Canonical options are now processed individually instead of one bulk `into` definition. Overall this was not a major update to the internals of handling the component model in Wasmtime. Instead this was mostly a surface-level refactoring to make sure that everything lines up with the new binary format for components. * All text syntax used in tests was updated to the new syntax. |
||
|
Alex Crichton
|
2af358dd9c |
Add a VMComponentContext type and create it on instantiation (#4215)
* Add a `VMComponentContext` type and create it on instantiation This commit fills out the `wasmtime-runtime` crate's support for `VMComponentContext` and creates it as part of the instantiation process. This moves a few maps that were temporarily allocated in an `InstanceData` into the `VMComponentContext` and additionally reads the canonical options data from there instead. This type still won't be used in its "full glory" until the lowering of host functions is completely implemented, however, which will be coming in a future commit. * Remove `DerefMut` implementation * Rebase conflicts |
||
|
Alex Crichton
|
3ed6fae7b3 |
Add trampoline compilation support for lowered imports (#4206)
* Add trampoline compilation support for lowered imports This commit adds support to the component model implementation for compiling trampolines suitable for calling host imports. Currently this is purely just the compilation side of things, modifying the wasmtime-cranelift crate and additionally filling out a new `VMComponentOffsets` type (similar to `VMOffsets`). The actual creation of a `VMComponentContext` is still not performed and will be a subsequent PR. Internally though some tests are actually possible with this where we at least assert that compilation of a component and creation of everything in-memory doesn't panic or trip any assertions, so some tests are added here for that as well. * Fix some test errors |
||
|
Alex Crichton
|
b49c5c878e |
Implement module imports into components (#4208)
* Implement module imports into components As a step towards implementing function imports into a component this commit implements importing modules into a component. This fills out missing pieces of functionality such as exporting modules as well. The previous translation code had initial support for translating imported modules but some of the AST type information was restructured with feedback from this implementation, namely splitting the `InstantiateModule` initializer into separate upvar/import variants to clarify that the item orderings for imports are resolved differently at runtime. Much of this commit is also adding infrastructure for any imports at all into a component. For example a `Linker` type (analagous to `wasmtime::Linker`) was added here as well. For now this type is quite limited due to the inability to define host functions (it can only work with instances and instances-of-modules) but it's enough to start writing `*.wast` tests which exercise lots of module-related functionality. * Fix a warning |
||
|
Alex Crichton
|
0cf0230432 |
Add dataflow processing to component translation for imports (#4205)
This commit enhances the processing of components to track all the dataflow for the processing of `canon.lower`'d functions. At the same time this fills out a few other missing details to component processing such as aliasing from some kinds of component instances and similar. The major changes contained within this are the updates the `info` submodule which has the AST of component type information. This has been significantly refactored to prepare for representing lowered functions and implementing those. The major change is from an `Instantiation` list to an `Initializer` list which abstractly represents a few other initialization actions. This work is split off from my main work to implement component imports of host functions. This is incomplete in the sense that it doesn't actually finish everything necessary to define host functions and import them into components. Instead this is only the changes necessary at the translation layer (so far). Consequently this commit does not have tests and also namely doesn't actually include the `VMComponentContext` initialization and usage. The full body of work is still a bit too messy to PR just yet so I'm hoping that this is a slimmed-down-enough piece to adequately be reviewed. |
||
|
Alex Crichton
|
140b83597b |
components: Implement the ability to call component exports (#4039)
* components: Implement the ability to call component exports This commit is an implementation of the typed method of calling component exports. This is intended to represent the most efficient way of calling a component in Wasmtime, similar to what `TypedFunc` represents today for core wasm. Internally this contains all the traits and implementations necessary to invoke component exports with any type signature (e.g. arbitrary parameters and/or results). The expectation is that for results we'll reuse all of this infrastructure except in reverse (arguments and results will be swapped when defining imports). Some features of this implementation are: * Arbitrary type hierarchies are supported * The Rust-standard `Option`, `Result`, `String`, `Vec<T>`, and tuple types all map down to the corresponding type in the component model. * Basic utf-16 string support is implemented as proof-of-concept to show what handling might look like. This will need further testing and benchmarking. * Arguments can be behind "smart pointers", so for example `&Rc<Arc<[u8]>>` corresponds to `list<u8>` in interface types. * Bulk copies from linear memory never happen unless explicitly instructed to do so. The goal of this commit is to create the ability to actually invoke wasm components. This represents what is expected to be the performance threshold for these calls where it ideally should be optimal how WebAssembly is invoked. One major missing piece of this is a `#[derive]` of some sort to generate Rust types for arbitrary `*.wit` types such as custom records, variants, flags, unions, etc. The current trait impls for tuples and `Result<T, E>` are expected to have fleshed out most of what such a derive would look like. There are some downsides and missing pieces to this commit and method of calling components, however, such as: * Passing `&[u8]` to WebAssembly is currently not optimal. Ideally this compiles down to a `memcpy`-equivalent somewhere but that currently doesn't happen due to all the bounds checks of copying data into memory. I have been unsuccessful so far at getting these bounds checks to be removed. * There is no finalization at this time (the "post return" functionality in the canonical ABI). Implementing this should be relatively straightforward but at this time requires `wasmparser` changes to catch up with the current canonical ABI. * There is no guarantee that results of a wasm function will be validated. As results are consumed they are validated but this means that if function returns an invalid string which the host doesn't look at then no trap will be generated. This is probably not the intended semantics of hosts in the component model. * At this time there's no support for memory64 memories, just a bunch of `FIXME`s to get around to. It's expected that this won't be too onerous, however. Some extra care will need to ensure that the various methods related to size/alignment all optimize to the same thing they do today (e.g. constants). * The return value of a typed component function is either `T` or `Value<T>`, and it depends on the ABI details of `T` and whether it takes up more than one return value slot or not. This is an ABI-implementation detail which is being forced through to the API layer which is pretty unfortunate. For example if you say the return value of a function is `(u8, u32)` then it's a runtime type-checking error. I don't know of a great way to solve this at this time. Overall I'm feeling optimistic about this trajectory of implementing value lifting/lowering in Wasmtime. While there are a number of downsides none seem completely insurmountable. There's naturally still a good deal of work with the component model but this should be a significant step up towards implementing and testing the component model. * Review comments * Write tests for calling functions This commit adds a new test file for actually executing functions and testing their results. This is not written as a `*.wast` test yet since it's not 100% clear if that's the best way to do that for now (given that dynamic signatures aren't supported yet). The tests themselves could all largely be translated to `*.wast` testing in the future, though, if supported. Along the way a number of minor issues were fixed with lowerings with the bugs exposed here. * Fix an endian mistake * Fix a typo and the `memory.fill` instruction |
||
|
Alex Crichton
|
fcf6208750 |
Initial skeleton of some component model processing (#4005)
* Initial skeleton of some component model processing This commit is the first of what will likely be many to implement the component model proposal in Wasmtime. This will be structured as a series of incremental commits, most of which haven't been written yet. My hope is to make this incremental and over time to make this easier to review and easier to test each step in isolation. Here much of the skeleton of how components are going to work in Wasmtime is sketched out. This is not a complete implementation of the component model so it's not all that useful yet, but some things you can do are: * Process the type section into a representation amenable for working with in Wasmtime. * Process the module section and register core wasm modules. * Process the instance section for core wasm modules. * Process core wasm module imports. * Process core wasm instance aliasing. * Ability to compile a component with core wasm embedded. * Ability to instantiate a component with no imports. * Ability to get functions from this component. This is already starting to diverge from the previous module linking representation where a `Component` will try to avoid unnecessary metadata about the component and instead internally only have the bare minimum necessary to instantiate the module. My hope is we can avoid constructing most of the index spaces during instantiation only for it to all ge thrown away. Additionally I'm predicting that we'll need to see through processing where possible to know how to generate adapters and where they are fused. At this time you can't actually call a component's functions, and that's the next PR that I would like to make. * Add tests for the component model support This commit uses the recently updated wasm-tools crates to add tests for the component model added in the previous commit. This involved updating the `wasmtime-wast` crate for component-model changes. Currently the component support there is quite primitive, but enough to at least instantiate components and verify the internals of Wasmtime are all working correctly. Additionally some simple tests for the embedding API have also been added. |