20f510671dbaa1c9c41b51f39f6c517461fc08d3
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Alex Crichton
|
b49c5c878e |
Implement module imports into components (#4208)
* Implement module imports into components As a step towards implementing function imports into a component this commit implements importing modules into a component. This fills out missing pieces of functionality such as exporting modules as well. The previous translation code had initial support for translating imported modules but some of the AST type information was restructured with feedback from this implementation, namely splitting the `InstantiateModule` initializer into separate upvar/import variants to clarify that the item orderings for imports are resolved differently at runtime. Much of this commit is also adding infrastructure for any imports at all into a component. For example a `Linker` type (analagous to `wasmtime::Linker`) was added here as well. For now this type is quite limited due to the inability to define host functions (it can only work with instances and instances-of-modules) but it's enough to start writing `*.wast` tests which exercise lots of module-related functionality. * Fix a warning |
||
|
Alex Crichton
|
d5ce51e8d1 |
Redesign interface type value representation (#4198)
Prior to this PR a major feature of calling component exports (#4039) was the usage of the `Value<T>` type. This type represents a value stored in wasm linear memory (the type `T` stored there). This implementation had a number of drawbacks though: * When returning a value it's ABI-specific whether you use `T` or `Value<T>` as a return value. If `T` is represented with one wasm primitive then you have to return `T`, otherwise the return value must be `Value<T>`. This is somewhat non-obvious and leaks ABI-details into the API which is unfortunate. * The `T` in `Value<T>` was somewhat non-obvious. For example a wasm-owned string was `Value<String>`. Using `Value<&str>` didn't work. * Working with `Value<T>` was unergonomic in the sense that you had to first "pair" it with a `&Store<U>` to get a `Cursor<T>` and then you could start reading the value. * Custom structs and enums, while not implemented yet, were planned to be quite wonky where when you had `Cursor<MyStruct>` then you would have to import a `CursorMyStructExt` trait generated by a proc-macro (think a `#[derive]` on the definition of `MyStruct`) which would enable field accessors, returning cursors of all the fields. * In general there was no "generic way" to load a `T` from memory. Other operations like lift/lower/store all had methods in the `ComponentValue` trait but load had no equivalent. None of these drawbacks were deal-breakers per-se. When I started to implement imported functions, though, the `Value<T>` type no longer worked. The major difference between imports and exports is that when receiving values from wasm an export returns at most one wasm primitive where an import can yield (through arguments) up to 16 wasm primitives. This means that if an export returned a string it would always be `Value<String>` but if an import took a string as an argument there was actually no way to represent this with `Value<String>` since the value wasn't actually stored in memory but rather the pointer/length pair is received as arguments. Overall this meant that `Value<T>` couldn't be used for arguments-to-imports, which means that altogether something new would be required. This PR completely removes the `Value<T>` and `Cursor<T>` type in favor of a different implementation. The inspiration from this comes from the fact that all primitives can be both lifted and lowered into wasm while it's just some times which can only go one direction. For example `String` can be lowered into wasm but can't be lifted from wasm. Instead some sort of "view" into wasm needs to be created during lifting. One of the realizations from #4039 was that we could leverage run-time-type-checking to reject static constructions that don't make sense. For example if an embedder asserts that a wasm function returns a Rust `String` we can reject that at typechecking time because it's impossible for a wasm module to ever do that. The new system of imports/exports in this PR now looks like: * Type-checking takes into accont an `Op` operation which indicates whether we'll be lifting or lowering the type. This means that we can allow the lowering operation for `String` but disallow the lifting operation. While we can't statically rule out an embedder saying that a component returns a `String` we can now reject it at runtime and disallow it from being called. * The `ComponentValue` trait now sports a new `load` function. This function will load and instance of `Self` from the byte-array provided. This is implemented for all types but only ever actually executed when the `lift` operation is allowed during type-checking. * The `Lift` associated type is removed since it's now expected that the lift operation returns `Self`. * The `ComponentReturn` trait is now no longer necessary and is removed. Instead returns are bounded by `ComponentValue`. During type-checking it's required that the return value can be lifted, disallowing, for example, returning a `String` or `&str`. * With `Value` gone there's no need to specify the ABI details of the return value, or whether it's communicated through memory or not. This means that handling return values through memory is transparently handled by Wasmtime. * Validation is in a sense more eagerly performed now. Whenever a value `T` is loaded the entire immediate structure of `T` is loaded and validated. Note that recursive through memory validation still does not happen, so the contents of lists or strings aren't validated, it's just validated that the pointers are in-bounds. Overall this felt like a much clearer system to work with and should be much easier to integrate with imported functions as well. The new `WasmStr` and `WasmList<T>` types can be used in import arguments and lifted from the immediate arguments provided rather than forcing them to always be stored in memory. |
||
|
Alex Crichton
|
140b83597b |
components: Implement the ability to call component exports (#4039)
* components: Implement the ability to call component exports This commit is an implementation of the typed method of calling component exports. This is intended to represent the most efficient way of calling a component in Wasmtime, similar to what `TypedFunc` represents today for core wasm. Internally this contains all the traits and implementations necessary to invoke component exports with any type signature (e.g. arbitrary parameters and/or results). The expectation is that for results we'll reuse all of this infrastructure except in reverse (arguments and results will be swapped when defining imports). Some features of this implementation are: * Arbitrary type hierarchies are supported * The Rust-standard `Option`, `Result`, `String`, `Vec<T>`, and tuple types all map down to the corresponding type in the component model. * Basic utf-16 string support is implemented as proof-of-concept to show what handling might look like. This will need further testing and benchmarking. * Arguments can be behind "smart pointers", so for example `&Rc<Arc<[u8]>>` corresponds to `list<u8>` in interface types. * Bulk copies from linear memory never happen unless explicitly instructed to do so. The goal of this commit is to create the ability to actually invoke wasm components. This represents what is expected to be the performance threshold for these calls where it ideally should be optimal how WebAssembly is invoked. One major missing piece of this is a `#[derive]` of some sort to generate Rust types for arbitrary `*.wit` types such as custom records, variants, flags, unions, etc. The current trait impls for tuples and `Result<T, E>` are expected to have fleshed out most of what such a derive would look like. There are some downsides and missing pieces to this commit and method of calling components, however, such as: * Passing `&[u8]` to WebAssembly is currently not optimal. Ideally this compiles down to a `memcpy`-equivalent somewhere but that currently doesn't happen due to all the bounds checks of copying data into memory. I have been unsuccessful so far at getting these bounds checks to be removed. * There is no finalization at this time (the "post return" functionality in the canonical ABI). Implementing this should be relatively straightforward but at this time requires `wasmparser` changes to catch up with the current canonical ABI. * There is no guarantee that results of a wasm function will be validated. As results are consumed they are validated but this means that if function returns an invalid string which the host doesn't look at then no trap will be generated. This is probably not the intended semantics of hosts in the component model. * At this time there's no support for memory64 memories, just a bunch of `FIXME`s to get around to. It's expected that this won't be too onerous, however. Some extra care will need to ensure that the various methods related to size/alignment all optimize to the same thing they do today (e.g. constants). * The return value of a typed component function is either `T` or `Value<T>`, and it depends on the ABI details of `T` and whether it takes up more than one return value slot or not. This is an ABI-implementation detail which is being forced through to the API layer which is pretty unfortunate. For example if you say the return value of a function is `(u8, u32)` then it's a runtime type-checking error. I don't know of a great way to solve this at this time. Overall I'm feeling optimistic about this trajectory of implementing value lifting/lowering in Wasmtime. While there are a number of downsides none seem completely insurmountable. There's naturally still a good deal of work with the component model but this should be a significant step up towards implementing and testing the component model. * Review comments * Write tests for calling functions This commit adds a new test file for actually executing functions and testing their results. This is not written as a `*.wast` test yet since it's not 100% clear if that's the best way to do that for now (given that dynamic signatures aren't supported yet). The tests themselves could all largely be translated to `*.wast` testing in the future, though, if supported. Along the way a number of minor issues were fixed with lowerings with the bugs exposed here. * Fix an endian mistake * Fix a typo and the `memory.fill` instruction |