* Add a `wasmtime::Linker` type
This commit adds a new type to the `wasmtime` crate, a `Linker`. This
linker is intended to vastly simplify calling `Instance::new` by easily
performing name resolution and incrementally defining state over time.
The goal here is to start down a path of making linking wasm modules in
`wasmtime` a first-class and ergonomic operation. This is highly likely
to evolve over time and get tweaked through releases as we iterate
towards a design well-suited for `wasmtime`, but this is intended to at
least be the initial foundation for such functionality.
This commit additionally also adds a C API for the linker and switches
the existing linking examples to using this linker in both Rust and C.
One piece of future work I'd like to tackle next is to integrate WASI
into the `wasmtime` crate in a more first-class manner. This [`Linker`]
type provides a great location to hook into the instantiation process to
easily instantiate modules with WASI imports. That's a relatively large
refactoring for now though and I figured it'd be best left for a
different time.
Closes#727
The crates/debug/src/transform/address_transform.rs is unoptimized in terms of data structures. This PR refactors this file to remove creation of intermediate in-heap structures, thus improves overall performance of the DWARF transformation.
* Reduce amount of memory allocated in translate_ranges_raw
* refactor translate_ranges
* Don't transform non-unit .debug_line
* type annotation for TransformRangeXXXIter's
* Fix empty generated wasm positions
Some structs and unions are large enough that making them `Copy` isn't
ideal. wasi-common only needed `Copy` in a few places that were easy to
fix. `SubscriptionClock` is 32 bytes, so it's not a bad a idea to pass
it by reference anyway.
* Add a .gitattributes file specifying LF-style line endings.
This is similar to [Rust's .gitattributes file] though simplified.
Most of our source and documentation files already used LF-style line
endings, including *.cs files, so this makes things more consistent.
[Rust's .gitattributes file]: https://github.com/rust-lang/rust/blob/master/.gitattributes
* Remove UTF-8 BOMs in *.cs files.
Most of our *.cs files don't have UTF-8 BOMs, so this makes things more
consistent.
* Remove the old wast and wasm2obj commands.
These are subsumed by the `wasmtime wast` and `wasmtime wasm2obj` commands.
Fixes#827.
* Remove wasm2obj install commands.
* Add examples of linking and WASI
This commit adds two example programs, one for linking two modules
together and one for instantiating WASI. The linkage example
additionally uses WASI to get some meaningful output at this time.
cc #1272
* Add examples to the book as well
* More links!
* Ignore examples from rustdoc testsing
* More example updates
* More ignored
with the prev approach, it would be passed by reference sometimes
(e.g. when used as an Array argument) but by value most of the time.
this was inconsistient.
theres no need to pass the owned version, all operations are &self.
* Use wiggle in place of wig in wasi-common
This is a rather massive commit that introduces `wiggle` into the
picture. We still use `wig`'s macro in `old` snapshot and to generate
`wasmtime-wasi` glue, but everything else is now autogenerated by `wiggle`.
In summary, thanks to `wiggle`, we no longer need to worry about
serialising and deserialising to and from the guest memory, and
all guest (WASI) types are now proper idiomatic Rust types.
While we're here, in preparation for the ephemeral snapshot, I went
ahead and reorganised the internal structure of the crate. Instead of
modules like `hostcalls_impl` or `hostcalls_impl::fs`, the structure
now resembles that in ephemeral with modules like `path`, `fd`, etc.
Now, I'm not requiring we leave it like this, but I reckon it looks
cleaner this way after all.
* Fix wig to use new first-class access to caller's mem
* Ignore warning in proc_exit for the moment
* Group unsafes together in args and environ calls
* Simplify pwrite; more unsafe blocks
* Simplify fd_read
* Bundle up unsafes in fd_readdir
* Simplify fd_write
* Add comment to path_readlink re zero-len buffers
* Simplify unsafes in random_get
* Hide GuestPtr<str> to &str in path::get
* Rewrite pread and pwrite using SeekFrom and read/write_vectored
I've left the implementation of VirtualFs pretty much untouched
as I don't feel that comfortable in changing the API too much.
Having said that, I reckon `pread` and `pwrite` could be refactored
out, and `preadv` and `pwritev` could be entirely rewritten using
`seek` and `read_vectored` and `write_vectored`.
* Add comment about VirtFs unsafety
* Fix all mentions of FdEntry to Entry
* Fix warnings on Win
* Add aux struct EntryTable responsible for Fds and Entries
This commit adds aux struct `EntryTable` which is private to `WasiCtx`
and is basically responsible for `Fd` alloc/dealloc as well as storing
matching `Entry`s. This struct is entirely private to `WasiCtx` and
as such as should remain transparent to `WasiCtx` users.
* Remove redundant check for empty buffer in path_readlink
* Preserve and rewind file cursor in pread/pwrite
* Use GuestPtr<[u8]>::copy_from_slice wherever copying bytes directly
* Use GuestPtr<[u8]>::copy_from_slice in fd_readdir
* Clean up unsafes around WasiCtx accessors
* Fix bugs in args_get and environ_get
* Fix conflicts after rebase
* Remove C++ dependency from `wasmtime`
This commit removes the last wads of C++ that we have in wasmtime,
meaning that building wasmtime no longer requires a C++ compiler. It
still does require a C toolchain for some minor purposes, but hopefully
we can remove that over time too!
The motivation for doing this is to consolidate all our signal-handling
code into one location in one language so you don't have to keep
crossing back and forth when understanding what's going on. This also
allows us to remove some extra cruft that wasn't necessary from the C++
original implementation. Additionally this should also make building
wasmtime a bit more portable since it's often easier to acquire a C
toolchain than it is to acquire a C++ toolchain. (e.g. if you're
cross-compiling to a musl target)
* Typos
* Enable jitdump profiling support by default
This the result of some of the investigation I was doing for #1017. I've
done a number of refactorings here which culminated in a number of
changes that all amount to what I think should result in jitdump support being
enabled by default:
* Pass in a list of finished functions instead of just a range to
ensure that we're emitting jit dump data for a specific module rather
than a whole `CodeMemory` which may have other modules.
* Define `ProfilingStrategy` in the `wasmtime` crate to have everything
locally-defined
* Add support to the C API to enable profiling
* Documentation added for profiling with jitdump to the book
* Split out supported/unsupported files in `jitdump.rs` to avoid having
lots of `#[cfg]`.
* Make dependencies optional that are only used for `jitdump`.
* Move initialization up-front to `JitDumpAgent::new()` instead of
deferring it to the first module.
* Pass around `Arc<dyn ProfilingAgent>` instead of
`Option<Arc<Mutex<Box<dyn ProfilingAgent>>>>`
The `jitdump` Cargo feature is now enabled by default which means that
our published binaries, C API artifacts, and crates will support
profiling at runtime by default. The support I don't think is fully
fleshed out and working but I think it's probably in a good enough spot
we can get users playing around with it!
Previously, we'd be very strict and disallow zero-length `wiggle_runtime::Region`s
altogether (we'd actually panic which is even worse). However, we
should allow this noting that any zero-length `Region` never
overlaps since its length is, well, zero. Additionally, this makes
`path_readlink` with zero buffers cleaner and possible without
additional checks/hacks around the passed in `GuestPtr<'_, [u8]>`
buffer.
* Remove `WrappedCallable` indirection
At this point `Func` has evolved quite a bit since inception and the
`WrappedCallable` trait I don't believe is needed any longer. This
should help clean up a few entry points by having fewer traits in play.
* Remove the `Callable` trait
This commit removes the `wasmtime::Callable` trait, changing the
signature of `Func::new` to take an appropriately typed `Fn`.
Additionally the function now always takes `&Caller` like `Func::wrap`
optionally can, to empower `Func::new` to have the same capabilities of
`Func::wrap`.
* Add a test for an already-fixed issue
Closes#849
* rustfmt
* Update more locations for `Callable`
* rustfmt
* Remove a stray leading borrow
* Review feedback
* Remove unneeded `wasmtime_call_trampoline` shim
* Add a first-class way of accessing caller's exports
This commit is a continuation of #1237 and updates the API of `Func` to
allow defining host functions which have easy access to a caller's
memory in particular. The new APIs look like so:
* The `Func::wrap*` family of functions was condensed into one
`Func::wrap` function.
* The ABI layer of conversions in `WasmTy` were removed
* An optional `Caller<'_>` argument can be at the front of all
host-defined functions now.
The old way the wasi bindings looked up memory has been removed and is
now replaced with the `Caller` type. The `Caller` type has a
`get_export` method on it which allows looking up a caller's export by
name, allowing you to get access to the caller's memory easily, and even
during instantiation.
* Add a temporary note
* Move some docs
- Convert recipes to have necessary size calculator
- Add a missing binemit function, `put_dynrexmp3`
- Modify the meta-encodings of x86 SIMD instructions to use `infer_rex()`, mostly through the `enc_both_inferred()` helper
- Fix up tests that previously always emitted a REX prefix
* Update wasi submodule
Removes some dependencies from the `witx` crate since WebAssembly/WASI#243
* Don't pull witx from two places
* Update submodule again
* Rename FdEntry to Entry
* Add custom FdSet container for managing fd allocs/deallocs
This commit adds a custom `FdSet` container which is intended
for use in `wasi-common` to track WASI fd allocs/deallocs. The
main aim for this container is to abstract away the current
approach of spawning new handles
```rust
fd = fd.checked_add(1).ok_or(...)?;
```
and to make it possible to reuse unused/reclaimed handles
which currently is not done.
The struct offers 3 methods to manage its functionality:
* `FdSet::new` initialises the internal data structures,
and most notably, it preallocates an `FdSet::BATCH_SIZE`
worth of handles in such a way that we always start popping
from the "smallest" handle (think of it as of reversed stack,
I guess; it's not a binary heap since we don't really care
whether internally the handles are sorted in some way, just that
the "largets" handle is at the bottom. Why will become clear
when describing `allocate` method.)
* `FdSet::allocate` pops the next available handle if one is available.
The tricky bit here is that, if we run out of handles, we preallocate
the next `FdSet::BATCH_SIZE` worth of handles starting from the
latest popped handle (i.e., the "largest" handle). This
works only because we make sure to only ever pop and push already
existing handles from the back, and push _new_ handles (from the
preallocation step) from the front. When we ultimately run out
of _all_ available handles, we then return `None` for the client
to handle in some way (e.g., throwing an error such as `WasiError::EMFILE`
or whatnot).
* `FdSet::deallocate` returns the already allocated handle back to
the pool for further reuse.
When figuring out the internals, I've tried to optimise for both
alloc and dealloc performance, and I believe we've got an amortised
`O(1)~*` performance for both (if my maths is right, and it may very
well not be, so please verify!).
In order to keep `FdSet` fairly generic, I've made sure not to hard-code
it for the current type system generated by `wig` (i.e., `wasi::__wasi_fd_t`
representing WASI handle), but rather, any type which wants to be managed
by `FdSet` needs to conform to `Fd` trait. This trait is quite simple as
it only requires a couple of rudimentary traits (although `std:#️⃣:Hash`
is quite a powerful assumption here!), and a custom method
```rust
Fd::next(&self) -> Option<Self>;
```
which is there to encapsulate creating another handle from the given one.
In the current state of the code, that'd be simply `u32::checked_add(1)`.
When `wiggle` makes it way into the `wasi-common`, I'd imagine it being
similar to
```rust
fn next(&self) -> Option<Self> {
self.0.checked_add(1).map(Self::from)
}
```
Anyhow, I'd be happy to learn your thoughts about this design!
* Fix compilation on other targets
* Rename FdSet to FdPool
* Fix FdPool unit tests
* Skip preallocation step in FdPool
* Replace 'replace' calls with direct assignment
* Reuse FdPool from snapshot1 in snapshot0
* Refactor FdPool::allocate
* Remove entry before deallocating the fd
* Refactor the design to accommodate `u32` as underlying type
This commit refactors the design by ensuring that the underlying
type in `FdPool` which we use to track and represent raw file
descriptors is `u32`. As a result, the structure of `FdPool` is
simplified massively as we no longer need to track the claimed
descriptors; in a way, we trust the caller to return the handle
after it's done with it. In case the caller decides to be clever
and return a handle which was not yet legally allocated, we panic.
This should never be a problem in `wasi-common` unless we hit a
bug.
To make all of this work, `Fd` trait is modified to require two
methods: `as_raw(&self) -> u32` and `from_raw(raw_fd: u32) -> Self`
both of which are used to convert to and from the `FdPool`'s underlying
type `u32`.
* Handle select relocations while generating trampolines
Trampoline generation for all function signatures exposed a preexisting
bug in wasmtime where trampoline generation occasionally does have
relocations, but it's asserted that trampolines don't generate
relocations, causing a panic. The relocation is currently primarily the
probestack function which happens when functions might have a huge
number of parameters, but not so huge as to blow the wasmparser limit of
how many parameters are allowed.
This commit fixes the issue by handling relocations for trampolines in
the same manner as the rest of the code. Note that dynamically-generated
trampolines via the `Func` API still panic if they have too many
arguments and generate a relocation, but it seems like we can try to fix
that later if the need truly arises.
Closes#1322
* Log trampoline relocations
Removes an edge towards the `synstructure` dependency in our dependency
graph which, if removed entirely, is hoped to reduce build times locally
and on CI as you switch between `cargo build` and `cargo test` because
deep dependencies like `syn` won't have their features alternating back
and forth.
* Build wasmtime-c-api differenty in run-examples
This tweaks how the wasmtime-c-api crate is built slightly, changing how
we invoke Cargo. Due to historical Cargo bugs this should help minimize
the number of rebuilds due to features since the feature selection will
be different.
* rustfmt
