* Publishing fixes.
* Make WASI a symlink.
* More fixes.
* Cargo doesn't allow dev-dependencies to have optional features.
* Remove the symlink.
* Add WASI as another git submodule.
* Shuffle around the wiggle crates
This commit reorganizes the wiggle crates slightly by performing the
following transforms:
* The `crates/wiggle` crate, previously named `wiggle`, was moved to
`crates/wiggle/crates/macro` and is renamed to `wiggle-macro`.
* The `crates/wiggle/crates/runtime` crate, previously named
`wiggle-runtime`, was moved to `crates/wiggle` and is renamed to
`wiggle`.
* The new `wiggle` crate depends on `wiggle-macro` and reexports the macro.
The goal here is that consumers only deal with the `wiggle` crate
itself. No more crates depend on `wiggle-runtime` and all dependencies
are entirely on just the `wiggle` crate.
* Remove the `crates/wiggle/crates` directory
Move everything into `crates/wiggle` directly, like `wasi-common`
* Add wiggle-macro to test-all script
* Fixup a test
Some structs and unions are large enough that making them `Copy` isn't
ideal. wasi-common only needed `Copy` in a few places that were easy to
fix. `SubscriptionClock` is 32 bytes, so it's not a bad a idea to pass
it by reference anyway.
* Use wiggle in place of wig in wasi-common
This is a rather massive commit that introduces `wiggle` into the
picture. We still use `wig`'s macro in `old` snapshot and to generate
`wasmtime-wasi` glue, but everything else is now autogenerated by `wiggle`.
In summary, thanks to `wiggle`, we no longer need to worry about
serialising and deserialising to and from the guest memory, and
all guest (WASI) types are now proper idiomatic Rust types.
While we're here, in preparation for the ephemeral snapshot, I went
ahead and reorganised the internal structure of the crate. Instead of
modules like `hostcalls_impl` or `hostcalls_impl::fs`, the structure
now resembles that in ephemeral with modules like `path`, `fd`, etc.
Now, I'm not requiring we leave it like this, but I reckon it looks
cleaner this way after all.
* Fix wig to use new first-class access to caller's mem
* Ignore warning in proc_exit for the moment
* Group unsafes together in args and environ calls
* Simplify pwrite; more unsafe blocks
* Simplify fd_read
* Bundle up unsafes in fd_readdir
* Simplify fd_write
* Add comment to path_readlink re zero-len buffers
* Simplify unsafes in random_get
* Hide GuestPtr<str> to &str in path::get
* Rewrite pread and pwrite using SeekFrom and read/write_vectored
I've left the implementation of VirtualFs pretty much untouched
as I don't feel that comfortable in changing the API too much.
Having said that, I reckon `pread` and `pwrite` could be refactored
out, and `preadv` and `pwritev` could be entirely rewritten using
`seek` and `read_vectored` and `write_vectored`.
* Add comment about VirtFs unsafety
* Fix all mentions of FdEntry to Entry
* Fix warnings on Win
* Add aux struct EntryTable responsible for Fds and Entries
This commit adds aux struct `EntryTable` which is private to `WasiCtx`
and is basically responsible for `Fd` alloc/dealloc as well as storing
matching `Entry`s. This struct is entirely private to `WasiCtx` and
as such as should remain transparent to `WasiCtx` users.
* Remove redundant check for empty buffer in path_readlink
* Preserve and rewind file cursor in pread/pwrite
* Use GuestPtr<[u8]>::copy_from_slice wherever copying bytes directly
* Use GuestPtr<[u8]>::copy_from_slice in fd_readdir
* Clean up unsafes around WasiCtx accessors
* Fix bugs in args_get and environ_get
* Fix conflicts after rebase
* Rename FdEntry to Entry
* Add custom FdSet container for managing fd allocs/deallocs
This commit adds a custom `FdSet` container which is intended
for use in `wasi-common` to track WASI fd allocs/deallocs. The
main aim for this container is to abstract away the current
approach of spawning new handles
```rust
fd = fd.checked_add(1).ok_or(...)?;
```
and to make it possible to reuse unused/reclaimed handles
which currently is not done.
The struct offers 3 methods to manage its functionality:
* `FdSet::new` initialises the internal data structures,
and most notably, it preallocates an `FdSet::BATCH_SIZE`
worth of handles in such a way that we always start popping
from the "smallest" handle (think of it as of reversed stack,
I guess; it's not a binary heap since we don't really care
whether internally the handles are sorted in some way, just that
the "largets" handle is at the bottom. Why will become clear
when describing `allocate` method.)
* `FdSet::allocate` pops the next available handle if one is available.
The tricky bit here is that, if we run out of handles, we preallocate
the next `FdSet::BATCH_SIZE` worth of handles starting from the
latest popped handle (i.e., the "largest" handle). This
works only because we make sure to only ever pop and push already
existing handles from the back, and push _new_ handles (from the
preallocation step) from the front. When we ultimately run out
of _all_ available handles, we then return `None` for the client
to handle in some way (e.g., throwing an error such as `WasiError::EMFILE`
or whatnot).
* `FdSet::deallocate` returns the already allocated handle back to
the pool for further reuse.
When figuring out the internals, I've tried to optimise for both
alloc and dealloc performance, and I believe we've got an amortised
`O(1)~*` performance for both (if my maths is right, and it may very
well not be, so please verify!).
In order to keep `FdSet` fairly generic, I've made sure not to hard-code
it for the current type system generated by `wig` (i.e., `wasi::__wasi_fd_t`
representing WASI handle), but rather, any type which wants to be managed
by `FdSet` needs to conform to `Fd` trait. This trait is quite simple as
it only requires a couple of rudimentary traits (although `std:#️⃣:Hash`
is quite a powerful assumption here!), and a custom method
```rust
Fd::next(&self) -> Option<Self>;
```
which is there to encapsulate creating another handle from the given one.
In the current state of the code, that'd be simply `u32::checked_add(1)`.
When `wiggle` makes it way into the `wasi-common`, I'd imagine it being
similar to
```rust
fn next(&self) -> Option<Self> {
self.0.checked_add(1).map(Self::from)
}
```
Anyhow, I'd be happy to learn your thoughts about this design!
* Fix compilation on other targets
* Rename FdSet to FdPool
* Fix FdPool unit tests
* Skip preallocation step in FdPool
* Replace 'replace' calls with direct assignment
* Reuse FdPool from snapshot1 in snapshot0
* Refactor FdPool::allocate
* Remove entry before deallocating the fd
* Refactor the design to accommodate `u32` as underlying type
This commit refactors the design by ensuring that the underlying
type in `FdPool` which we use to track and represent raw file
descriptors is `u32`. As a result, the structure of `FdPool` is
simplified massively as we no longer need to track the claimed
descriptors; in a way, we trust the caller to return the handle
after it's done with it. In case the caller decides to be clever
and return a handle which was not yet legally allocated, we panic.
This should never be a problem in `wasi-common` unless we hit a
bug.
To make all of this work, `Fd` trait is modified to require two
methods: `as_raw(&self) -> u32` and `from_raw(raw_fd: u32) -> Self`
both of which are used to convert to and from the `FdPool`'s underlying
type `u32`.
* Introduce WasiCtxBuilderError error type
`WasiCtxBuilderError` is the `wasi-common` client-facing error type
which is exclusively thrown when building a new `WasiCtx` instance.
As such, building such an instance should not require the client to
understand different WASI errno values as was assumed until now.
This commit is a first step at streamlining error handling in
`wasi-common` and makes way for the `wiggle` crate.
When adding the `WasiCtxBuilderError`, I've had to do two things of
notable importance:
1. I've removed a couple of `ok_or` calls in `WasiCtxBuilder::build`
and replaced them with `unwrap`s, following the same pattern in
different builder methods above. This is fine since we _always_
operate on non-empty `Option`s in `WasiCtxBuilder` thus `unwrap`ing
will never fail. On the other hand, this might be a good opportunity
to rethink the structure of our builder, and how we good remove
the said `Option`s especially since we always populate them with
empty containers to begin with. I understand this is to make
chaining of builder methods easier which take and return `&mut self`
and the same applies to `WasiCtxBuilder::build(&mut self)` method,
but perhaps it would more cleanly signal the intentions if we simply
moved `WasiCtxBuilder` instance around. Food for thought!
2. Methods specific to determining rights of passed around `std::fs::File`
objects when populating `WasiCtx` `FdEntry` entities now return
`io::Error` directly so that we can reuse them in `WasiCtxBuilder` methods
(returning `WasiCtxBuilderError` error type), and in syscalls
(returning WASI errno).
* Return WasiError directly in syscalls
Also, removes `error::Error` type altogether. Now, `io::Error` and
related are automatically converted to their corresponding WASI
errno value encapsulated as `WasiError`.
While here, it made sense to me to move `WasiError` to `wasi` module
which will align itself well with the upcoming changes introduced
by `wiggle`. To different standard `Result` from WASI specific, I've
created a helper alias `WasiResult` also residing in `wasi` module.
* Update wig
* Add from ffi::NulError and pass context to NotADirectory
* Add dummy commit to test CI
* witx tagged unions: updates to wig to use new semantics
* wig: emit a `#variant: ()` union variant for empty variants
* wasi-common: translate to use tagged unions
* update to flattened layout of event struct
* wig: generate layout tests, and delete bindgen ones
the bindgen tests became out-of-date with the latest changes to the
representation of unions, and the re-jiggering of various struct
definitions that went along with it.
* wasi: point at master with tagged union PR merged
* fix event struct repr on windows
This commit introduces two small changes:
* it adds `gen_errno_strerror` to `wig` crate which generates a
`strerror` function for `__wasi_errno_t` directly from `*.witx`,
similarly to how it's done in the `wasi` crate
* it tweaks `WasiError` type to include the error message generated
with `strerror` when displaying the error
* Add support for wasi_snapshot_preview1.
This adds support for the new ABI, while preserving compatibility
support for the old ABI.
* Fix compilation on platforms where nlink_t isn't 64-bit.
* rustfmt
* Fix Windows build errors.
