* Refactor where results of compilation are stored
This commit refactors the internals of compilation in Wasmtime to change
where results of individual function compilation are stored. Previously
compilation resulted in many maps being returned, and compilation
results generally held all these maps together. This commit instead
switches this to have all metadata stored in a `CompiledFunction`
instead of having a separate map for each item that can be stored.
The motivation for this is primarily to help out with future
module-linking-related PRs. What exactly "module level" is depends on
how we interpret modules and how many modules are in play, so it's a bit
easier for operations in wasmtime to work at the function level where
possible. This means that we don't have to pass around multiple
different maps and a function index, but instead just one map or just
one entry representing a compiled function.
Additionally this change updates where the parallelism of compilation
happens, pushing it into `wasmtime-jit` instead of `wasmtime-environ`.
This is another goal where `wasmtime-jit` will have more knowledge about
module-level pieces with module linking in play. User-facing-wise this
should be the same in terms of parallel compilation, though.
The ultimate goal of this refactoring is to make it easier for the
results of compilation to actually be a set of wasm modules. This means
we won't be able to have a map-per-metadata where the primary key is the
function index, because there will be many modules within one "object
file".
* Don't clear out fields, just don't store them
Persist a smaller set of fields in `CompilationArtifacts` instead of
trying to clear fields out and dynamically not accessing them.
* Don't re-parse wasm for debuginfo
This commit updates debuginfo parsing to happen during the main
translation of the original wasm module. This avoid re-parsing the wasm
module twice (at least the section-level headers). Additionally this
ties debuginfo directly to a `ModuleTranslation` which makes it easier
to process debuginfo for nested modules in the upcoming module linking
proposal.
The changes here are summarized by taking the `read_debuginfo` function
and merging it with the main module translation that happens which is
driven by cranelift. Some new hooks were added to the module environment
trait to support this, but most of it was integrating with existing hooks.
* Fix tests in debug crate
* virtfs file: update cursor position on fd_read
If a handle is backed by InMemoryFile, fd_read (turned into
Handle::read_vectored) doesn't update the cursor position properly and
thus prevents the caller from detecting EOF.
* virtfs file: fd_{pread,pwrite}: update offset in iovec iteration
If multiple iovec's are supplied, fd_pread and fd_pwrite previously
access data at the same offset for each iovec.
* move caching to the CompilationArtifacts
* mv cache_config from Compiler to CompiledModule
* hash isa flags
* no cache for wasm2obj
* mv caching to wasmtime crate
* account each Compiler field when hash
* Use AsRef<Path> instead of AsRef<OsStr> in yanix functions.
`AsRef<Path>` makes these more consistent with `std` interfaces, making
them easier to use outside of wasi-common.
Also, refactor the conversion to `CString` into a helper function.
* Reduce clutter from fully-qualifying names.
* rustfmt
This commit updates our CI to verify that all crates are publish-able at
all times on every commit. During the 0.19.0 release we found another
case where the crates as they live in this repository weren't
publish-able, so the hope is that this no longer comes up again!
The script added in this commit also takes the time/liberty to remove
the existing bump/publish scripts and instead replace them with one Rust
script originally sourced from wasm-bindgen. The intention of this
script is that it has three modes:
* `./publish bump` - bumps version numbers which are sent as a PR to get
reviewed (probably with a changelog as well)
* `./publish verify` - run on CI on every commit, builds every crate we
publish as if it's being published to crates.io, notably without raw
access to other crates in the repository.
* `./publish publish` - publishes all crates to crates.io, passing the
`--no-verify` flag to make this a much speedier process than it is
today.
This commit fixes `Clone for wasm_val_t` to avoid attempting to chase a
null pointer. It also fixes the implementation for `FuncRef` values by
cloning their internal `wasm_ref_t` as well.
Currently `Func::new` will panic if one of the arguments of the function
is a reference type and the `Store` doesn't have reference types
enabled. This happens because cranelift isn't configure to enable stack
maps but the register allocators expects them to exist when reference
types are seen.
The fix here is to always enable reference types in cranelift for our
trampoline generation and `Func::new`. This should hopefully ensure that
trampolines are generated correctly and they'll just not be able to get
hooked up to an `Instance` because validation will prevent reference
types from being used elsewhere.
* wasmtime-c-api: Only drop non-null `*mut wasm_ref_t`s
* wasmtime-c-api: Handle null refs in `wasm_val_t` to `Val` conversion
* wasmtime-c-api: Don't unwrap and rewrap `Option`s
The `unwrap` can panic, and there isn't any point to this unwrap+rewrap.
* wasmtime-c-api: Add conversions between `funcref` and `wasm_func_t`
* wasmtime-c-api: More ownership documentation for `wasmtime.h`
Before this patch, running the x64 new backend would require both
compiling with --features experimental_x64 and running with
`use_new_backend`.
This patches changes this behavior so that the runtime flag is not
needed anymore: using the feature flag will enforce usage of the new
backend everywhere, making using and testing it much simpler:
cargo run --features experimental_x64 ;; other CLI options/flags
This also gives a hint at what the meta language generation would look
like after switching to the new backend.
Compiling only with the x64 codegen flag gives a nice compile time speedup.
This commit is intended to update wasmparser to 0.59.0. This primarily
includes bytecodealliance/wasm-tools#40 which is a large update to how
parsing and validation works. The impact on Wasmtime is pretty small at
this time, but over time I'd like to refactor the internals here to lean
more heavily on that upstream wasmparser refactoring.
For now, though, the intention is to get on the train of wasmparser's
latest `main` branch to ensure we get bug fixes and such.
As part of this update a few other crates and such were updated. This is
primarily to handle the new encoding of `ref.is_null` where the type is
not part of the instruction encoding any more.
We were accidentally always running the `fib-debug/main` example because of
shenanigans with alphabetical ordering and hard coding "main.exe" as the command
we run. Now we properly detect which example we built and run the appropriate
executable.
The canonical examples tests will be the examples run via `cargo run -p
run-examples` from now on. Those tests have two advantages over the ones being
deleted:
1. They will automatically pick up new examples and run/test them.
2. They support all of Linux, MacOS, and Windows, while the tests being deleted
are Linux only.
This required that `wasm_val_t` have a `Drop` implementation, an explicit
`Clone` implementation, and no longer be `Copy`, which rippled out through the
crate a bit.
Additionally, `wasm_func_call` and friends were creating references to
uninitialized data for its out pointers and assigning to them. As soon as
`wasm_val_t` gained a `Drop` impl and tried to drop the old value of the
assignment (which is uninitialized data), then things blew up. The fix is to
properly represent the out pointers with `MaybeUninit`, and use `ptr::write` to
initialize them without dropping the old data.
Part of #929
- Create the ELF image from Compilation
- Create CodeMemory from the ELF image
- Link using ELF image
- Remove creation of GDB JIT images from crates/debug
- Move make_trampoline from compiler.rs
