T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

wasi-threads: an initial implementation (#5484)

Browse Source 
This commit includes a set of changes that add initial support for `wasi-threads` to Wasmtime:

* feat: remove mutability from the WasiCtx Table

This patch adds interior mutability to the WasiCtx Table and the Table elements.

Major pain points:
* `File` only needs `RwLock<cap_std::fs::File>` to implement
  `File::set_fdflags()` on Windows, because of [1]
* Because `File` needs a `RwLock` and `RwLock*Guard` cannot
  be hold across an `.await`, The `async` from
  `async fn num_ready_bytes(&self)` had to be removed
* Because `File` needs a `RwLock` and `RwLock*Guard` cannot
  be dereferenced in `pollable`, the signature of
  `fn pollable(&self) -> Option<rustix::fd::BorrowedFd>`
  changed to `fn pollable(&self) -> Option<Arc<dyn AsFd + '_>>`

[1] da238e324e/src/fs/fd_flags.rs (L210-L217)

* wasi-threads: add an initial implementation

This change is a first step toward implementing `wasi-threads` in
Wasmtime. We may find that it has some missing pieces, but the core
functionality is there: when `wasi::thread_spawn` is called by a running
WebAssembly module, a function named `wasi_thread_start` is found in the
module's exports and called in a new instance. The shared memory of the
original instance is reused in the new instance.

This new WASI proposal is in its early stages and details are still
being hashed out in the [spec] and [wasi-libc] repositories. Due to its
experimental state, the `wasi-threads` functionality is hidden behind
both a compile-time and runtime flag: one must build with `--features
wasi-threads` but also run the Wasmtime CLI with `--wasm-features
threads` and `--wasi-modules experimental-wasi-threads`. One can
experiment with `wasi-threads` by running:

```console
$ cargo run --features wasi-threads -- \
    --wasm-features threads --wasi-modules experimental-wasi-threads \
    <a threads-enabled module>
```

Threads-enabled Wasm modules are not yet easy to build. Hopefully this
is resolved soon, but in the meantime see the use of
`THREAD_MODEL=posix` in the [wasi-libc] repository for some clues on
what is necessary. Wiggle complicates things by requiring the Wasm
memory to be exported with a certain name and `wasi-threads` also
expects that memory to be imported; this build-time obstacle can be
overcome with the `--import-memory --export-memory` flags only available
in the latest Clang tree. Due to all of this, the included tests are
written directly in WAT--run these with:

```console
$ cargo test --features wasi-threads -p wasmtime-cli -- cli_tests
```

[spec]: https://github.com/WebAssembly/wasi-threads
[wasi-libc]: https://github.com/WebAssembly/wasi-libc

This change does not protect the WASI implementations themselves from
concurrent access. This is already complete in previous commits or left
for future commits in certain cases (e.g., wasi-nn).

* wasi-threads: factor out process exit logic

As is being discussed [elsewhere], either calling `proc_exit` or
trapping in any thread should halt execution of all threads. The
Wasmtime CLI already has logic for adapting a WebAssembly error code to
a code expected in each OS. This change factors out this logic to a new
function, `maybe_exit_on_error`, for use within the `wasi-threads`
implementation.

This will work reasonably well for CLI users of Wasmtime +
`wasi-threads`, but embedders will want something better in the future:
when a `wasi-threads` threads fails, they may not want their application
to exit. Handling this is tricky, because it will require cancelling the
threads spawned by the `wasi-threads` implementation, something that is
not trivial to do in Rust. With this change, we defer that work until
later in order to provide a working implementation of `wasi-threads` for
experimentation.

[elsewhere]: https://github.com/WebAssembly/wasi-threads/pull/17

* review: work around `fd_fdstat_set_flags`

In order to make progress with wasi-threads, this change temporarily
works around limitations induced by `wasi-common`'s
`fd_fdstat_set_flags` to allow `&mut self` use in the implementation.
Eventual resolution is tracked in
https://github.com/bytecodealliance/wasmtime/issues/5643. This change
makes several related helper functions (e.g., `set_fdflags`) take `&mut
self` as well.

* test: use `wait`/`notify` to improve `threads.wat` test

Previously, the test simply executed in a loop for some hardcoded number
of iterations. This changes uses `wait` and `notify` and atomic
operations to keep track of when the spawned threads are done and join
on the main thread appropriately.

* various fixes and tweaks due to the PR review

---------

Signed-off-by: Harald Hoyer <harald@profian.com>
Co-authored-by: Harald Hoyer <harald@profian.com>
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
This commit is contained in:
Andrew Brown
2023-02-07 13:43:02 -08:00
committed by GitHub
parent 2c8425998b
commit edfa10d607
33 changed files with 869 additions and 427 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
Cargo.lock generated
View File

@@ -3149,6 +3149,7 @@ dependencies = [
"cap-rand", "cap-rand",
"cap-std", "cap-std",
"io-extras", "io-extras",
"log",
"rustix", "rustix",
"thiserror", "thiserror",
"tracing", "tracing",
@@ -3506,6 +3507,7 @@ dependencies = [
"wasmtime-wasi", "wasmtime-wasi",
"wasmtime-wasi-crypto", "wasmtime-wasi-crypto",
"wasmtime-wasi-nn", "wasmtime-wasi-nn",
"wasmtime-wasi-threads",
"wasmtime-wast", "wasmtime-wast",
"wast 52.0.2", "wast 52.0.2",
"wat", "wat",
@@ -3746,6 +3748,7 @@ name = "wasmtime-wasi"
version = "7.0.0" version = "7.0.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"libc",
"wasi-cap-std-sync", "wasi-cap-std-sync",
"wasi-common", "wasi-common",
"wasi-tokio", "wasi-tokio",
@@ -3774,6 +3777,18 @@ dependencies = [
"wiggle", "wiggle",
] ]
[[package]]
name = "wasmtime-wasi-threads"
version = "7.0.0"
dependencies = [
"anyhow",
"log",
"rand 0.8.5",
"wasi-common",
"wasmtime",
"wasmtime-wasi",
]
[[package]] [[package]]
name = "wasmtime-wast" name = "wasmtime-wast"
version = "7.0.0" version = "7.0.0"

7
Cargo.toml
View File

@@ -28,13 +28,13 @@ wasmtime-cli-flags = { workspace = true }
wasmtime-cranelift = { workspace = true } wasmtime-cranelift = { workspace = true }
wasmtime-environ = { workspace = true } wasmtime-environ = { workspace = true }
wasmtime-wast = { workspace = true } wasmtime-wast = { workspace = true }
wasmtime-wasi = { workspace = true } wasmtime-wasi = { workspace = true, features = ["exit"] }
wasmtime-wasi-crypto = { workspace = true, optional = true } wasmtime-wasi-crypto = { workspace = true, optional = true }
wasmtime-wasi-nn = { workspace = true, optional = true } wasmtime-wasi-nn = { workspace = true, optional = true }
wasmtime-wasi-threads = { workspace = true, optional = true }
clap = { workspace = true, features = ["color", "suggestions", "derive"] } clap = { workspace = true, features = ["color", "suggestions", "derive"] }
anyhow = { workspace = true } anyhow = { workspace = true }
target-lexicon = { workspace = true } target-lexicon = { workspace = true }
libc = "0.2.60"
humantime = "2.0.0" humantime = "2.0.0"
once_cell = { workspace = true } once_cell = { workspace = true }
listenfd = "1.0.0" listenfd = "1.0.0"
@@ -68,6 +68,7 @@ wasmtime-component-util = { workspace = true }
component-macro-test = { path = "crates/misc/component-macro-test" } component-macro-test = { path = "crates/misc/component-macro-test" }
component-test-util = { workspace = true } component-test-util = { workspace = true }
bstr = "0.2.17" bstr = "0.2.17"
libc = "0.2.60"
[target.'cfg(windows)'.dev-dependencies] [target.'cfg(windows)'.dev-dependencies]
windows-sys = { workspace = true, features = ["Win32_System_Memory"] } windows-sys = { workspace = true, features = ["Win32_System_Memory"] }
@@ -124,6 +125,7 @@ wasmtime-wast = { path = "crates/wast", version = "=7.0.0" }
wasmtime-wasi = { path = "crates/wasi", version = "7.0.0" } wasmtime-wasi = { path = "crates/wasi", version = "7.0.0" }
wasmtime-wasi-crypto = { path = "crates/wasi-crypto", version = "7.0.0" } wasmtime-wasi-crypto = { path = "crates/wasi-crypto", version = "7.0.0" }
wasmtime-wasi-nn = { path = "crates/wasi-nn", version = "7.0.0" } wasmtime-wasi-nn = { path = "crates/wasi-nn", version = "7.0.0" }
wasmtime-wasi-threads = { path = "crates/wasi-threads", version = "7.0.0" }
wasmtime-component-util = { path = "crates/component-util", version = "=7.0.0" } wasmtime-component-util = { path = "crates/component-util", version = "=7.0.0" }
wasmtime-component-macro = { path = "crates/component-macro", version = "=7.0.0" } wasmtime-component-macro = { path = "crates/component-macro", version = "=7.0.0" }
wasmtime-asm-macros = { path = "crates/asm-macros", version = "=7.0.0" } wasmtime-asm-macros = { path = "crates/asm-macros", version = "=7.0.0" }
@@ -205,6 +207,7 @@ jitdump = ["wasmtime/jitdump"]
vtune = ["wasmtime/vtune"] vtune = ["wasmtime/vtune"]
wasi-crypto = ["dep:wasmtime-wasi-crypto"] wasi-crypto = ["dep:wasmtime-wasi-crypto"]
wasi-nn = ["dep:wasmtime-wasi-nn"] wasi-nn = ["dep:wasmtime-wasi-nn"]
wasi-threads = ["dep:wasmtime-wasi-threads"]
pooling-allocator = ["wasmtime/pooling-allocator", "wasmtime-cli-flags/pooling-allocator"] pooling-allocator = ["wasmtime/pooling-allocator", "wasmtime-cli-flags/pooling-allocator"]
all-arch = ["wasmtime/all-arch"] all-arch = ["wasmtime/all-arch"]
posix-signals-on-macos = ["wasmtime/posix-signals-on-macos"] posix-signals-on-macos = ["wasmtime/posix-signals-on-macos"]

1
ci/run-tests.sh
View File

@@ -2,6 +2,7 @@
cargo test \ cargo test \
--features "test-programs/test_programs" \ --features "test-programs/test_programs" \
--features wasi-threads \
--workspace \ --workspace \
--exclude 'wasmtime-wasi-*' \ --exclude 'wasmtime-wasi-*' \
--exclude wasi-crypto \ --exclude wasi-crypto \

36
crates/cli-flags/src/lib.rs
View File

@@ -50,13 +50,17 @@ pub const SUPPORTED_WASI_MODULES: &[(&str, &str)] = &[
"wasi-common", "wasi-common",
"enables support for the WASI common APIs, see https://github.com/WebAssembly/WASI", "enables support for the WASI common APIs, see https://github.com/WebAssembly/WASI",
), ),
(
"experimental-wasi-crypto",
"enables support for the WASI cryptography APIs (experimental), see https://github.com/WebAssembly/wasi-crypto",
),
( (
"experimental-wasi-nn", "experimental-wasi-nn",
"enables support for the WASI neural network API (experimental), see https://github.com/WebAssembly/wasi-nn", "enables support for the WASI neural network API (experimental), see https://github.com/WebAssembly/wasi-nn",
), ),
( (
"experimental-wasi-crypto", "experimental-wasi-threads",
"enables support for the WASI cryptography APIs (experimental), see https://github.com/WebAssembly/wasi-crypto", "enables support for the WASI threading API (experimental), see https://github.com/WebAssembly/wasi-threads",
), ),
]; ];
@@ -466,8 +470,9 @@ fn parse_wasi_modules(modules: &str) -> Result<WasiModules> {
let mut set = |module: &str, enable: bool| match module { let mut set = |module: &str, enable: bool| match module {
"" => Ok(()), "" => Ok(()),
"wasi-common" => Ok(wasi_modules.wasi_common = enable), "wasi-common" => Ok(wasi_modules.wasi_common = enable),
"experimental-wasi-nn" => Ok(wasi_modules.wasi_nn = enable),
"experimental-wasi-crypto" => Ok(wasi_modules.wasi_crypto = enable), "experimental-wasi-crypto" => Ok(wasi_modules.wasi_crypto = enable),
"experimental-wasi-nn" => Ok(wasi_modules.wasi_nn = enable),
"experimental-wasi-threads" => Ok(wasi_modules.wasi_threads = enable),
"default" => bail!("'default' cannot be specified with other WASI modules"), "default" => bail!("'default' cannot be specified with other WASI modules"),
_ => bail!("unsupported WASI module '{}'", module), _ => bail!("unsupported WASI module '{}'", module),
}; };
@@ -494,19 +499,23 @@ pub struct WasiModules {
/// parts once the implementation allows for it (e.g. wasi-fs, wasi-clocks, etc.). /// parts once the implementation allows for it (e.g. wasi-fs, wasi-clocks, etc.).
pub wasi_common: bool, pub wasi_common: bool,
/// Enable the experimental wasi-nn implementation /// Enable the experimental wasi-crypto implementation.
pub wasi_crypto: bool,
/// Enable the experimental wasi-nn implementation.
pub wasi_nn: bool, pub wasi_nn: bool,
/// Enable the experimental wasi-crypto implementation /// Enable the experimental wasi-threads implementation.
pub wasi_crypto: bool, pub wasi_threads: bool,
} }
impl Default for WasiModules { impl Default for WasiModules {
fn default() -> Self { fn default() -> Self {
Self { Self {
wasi_common: true, wasi_common: true,
wasi_nn: false,
wasi_crypto: false, wasi_crypto: false,
wasi_nn: false,
wasi_threads: false,
} }
} }
} }
@@ -518,6 +527,7 @@ impl WasiModules {
wasi_common: false, wasi_common: false,
wasi_nn: false, wasi_nn: false,
wasi_crypto: false, wasi_crypto: false,
wasi_threads: false,
} }
} }
} }
@@ -663,8 +673,9 @@ mod test {
options.wasi_modules.unwrap(), options.wasi_modules.unwrap(),
WasiModules { WasiModules {
wasi_common: true, wasi_common: true,
wasi_crypto: false,
wasi_nn: false, wasi_nn: false,
wasi_crypto: false wasi_threads: false
} }
); );
} }
@@ -676,8 +687,9 @@ mod test {
options.wasi_modules.unwrap(), options.wasi_modules.unwrap(),
WasiModules { WasiModules {
wasi_common: true, wasi_common: true,
wasi_crypto: false,
wasi_nn: false, wasi_nn: false,
wasi_crypto: false wasi_threads: false
} }
); );
} }
@@ -693,8 +705,9 @@ mod test {
options.wasi_modules.unwrap(), options.wasi_modules.unwrap(),
WasiModules { WasiModules {
wasi_common: false, wasi_common: false,
wasi_crypto: false,
wasi_nn: true, wasi_nn: true,
wasi_crypto: false wasi_threads: false
} }
); );
} }
@@ -707,8 +720,9 @@ mod test {
options.wasi_modules.unwrap(), options.wasi_modules.unwrap(),
WasiModules { WasiModules {
wasi_common: false, wasi_common: false,
wasi_crypto: false,
wasi_nn: false, wasi_nn: false,
wasi_crypto: false wasi_threads: false
} }
); );
} }

1
crates/wasi-common/Cargo.toml
View File

@@ -26,6 +26,7 @@ tracing = { workspace = true }
cap-std = { workspace = true } cap-std = { workspace = true }
cap-rand = { workspace = true } cap-rand = { workspace = true }
bitflags = { workspace = true } bitflags = { workspace = true }
log = { workspace = true }
[target.'cfg(unix)'.dependencies] [target.'cfg(unix)'.dependencies]
rustix = { workspace = true, features = ["fs"] } rustix = { workspace = true, features = ["fs"] }

35
crates/wasi-common/cap-std-sync/src/file.rs
View File

@@ -31,24 +31,23 @@ impl WasiFile for File {
fn pollable(&self) -> Option<rustix::fd::BorrowedFd> { fn pollable(&self) -> Option<rustix::fd::BorrowedFd> {
Some(self.0.as_fd()) Some(self.0.as_fd())
} }
#[cfg(windows)] #[cfg(windows)]
fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> { fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> {
Some(self.0.as_raw_handle_or_socket()) Some(self.0.as_raw_handle_or_socket())
} }
async fn datasync(&mut self) -> Result<(), Error> { async fn datasync(&self) -> Result<(), Error> {
self.0.sync_data()?; self.0.sync_data()?;
Ok(()) Ok(())
} }
async fn sync(&mut self) -> Result<(), Error> { async fn sync(&self) -> Result<(), Error> {
self.0.sync_all()?; self.0.sync_all()?;
Ok(()) Ok(())
} }
async fn get_filetype(&mut self) -> Result<FileType, Error> { async fn get_filetype(&self) -> Result<FileType, Error> {
let meta = self.0.metadata()?; let meta = self.0.metadata()?;
Ok(filetype_from(&meta.file_type())) Ok(filetype_from(&meta.file_type()))
} }
async fn get_fdflags(&mut self) -> Result<FdFlags, Error> { async fn get_fdflags(&self) -> Result<FdFlags, Error> {
let fdflags = get_fd_flags(&self.0)?; let fdflags = get_fd_flags(&self.0)?;
Ok(fdflags) Ok(fdflags)
} }
@@ -64,7 +63,7 @@ impl WasiFile for File {
self.0.set_fd_flags(set_fd_flags)?; self.0.set_fd_flags(set_fd_flags)?;
Ok(()) Ok(())
} }
async fn get_filestat(&mut self) -> Result<Filestat, Error> { async fn get_filestat(&self) -> Result<Filestat, Error> {
let meta = self.0.metadata()?; let meta = self.0.metadata()?;
Ok(Filestat { Ok(Filestat {
device_id: meta.dev(), device_id: meta.dev(),
@@ -77,20 +76,20 @@ impl WasiFile for File {
ctim: meta.created().map(|t| Some(t.into_std())).unwrap_or(None), ctim: meta.created().map(|t| Some(t.into_std())).unwrap_or(None),
}) })
} }
async fn set_filestat_size(&mut self, size: u64) -> Result<(), Error> { async fn set_filestat_size(&self, size: u64) -> Result<(), Error> {
self.0.set_len(size)?; self.0.set_len(size)?;
Ok(()) Ok(())
} }
async fn advise(&mut self, offset: u64, len: u64, advice: Advice) -> Result<(), Error> { async fn advise(&self, offset: u64, len: u64, advice: Advice) -> Result<(), Error> {
self.0.advise(offset, len, convert_advice(advice))?; self.0.advise(offset, len, convert_advice(advice))?;
Ok(()) Ok(())
} }
async fn allocate(&mut self, offset: u64, len: u64) -> Result<(), Error> { async fn allocate(&self, offset: u64, len: u64) -> Result<(), Error> {
self.0.allocate(offset, len)?; self.0.allocate(offset, len)?;
Ok(()) Ok(())
} }
async fn set_times( async fn set_times(
&mut self, &self,
atime: Option<wasi_common::SystemTimeSpec>, atime: Option<wasi_common::SystemTimeSpec>,
mtime: Option<wasi_common::SystemTimeSpec>, mtime: Option<wasi_common::SystemTimeSpec>,
) -> Result<(), Error> { ) -> Result<(), Error> {
@@ -98,41 +97,41 @@ impl WasiFile for File {
.set_times(convert_systimespec(atime), convert_systimespec(mtime))?; .set_times(convert_systimespec(atime), convert_systimespec(mtime))?;
Ok(()) Ok(())
} }
async fn read_vectored<'a>(&mut self, bufs: &mut [io::IoSliceMut<'a>]) -> Result<u64, Error> { async fn read_vectored<'a>(&self, bufs: &mut [io::IoSliceMut<'a>]) -> Result<u64, Error> {
let n = self.0.read_vectored(bufs)?; let n = self.0.read_vectored(bufs)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }
async fn read_vectored_at<'a>( async fn read_vectored_at<'a>(
&mut self, &self,
bufs: &mut [io::IoSliceMut<'a>], bufs: &mut [io::IoSliceMut<'a>],
offset: u64, offset: u64,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
let n = self.0.read_vectored_at(bufs, offset)?; let n = self.0.read_vectored_at(bufs, offset)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }
async fn write_vectored<'a>(&mut self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> { async fn write_vectored<'a>(&self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> {
let n = self.0.write_vectored(bufs)?; let n = self.0.write_vectored(bufs)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }
async fn write_vectored_at<'a>( async fn write_vectored_at<'a>(
&mut self, &self,
bufs: &[io::IoSlice<'a>], bufs: &[io::IoSlice<'a>],
offset: u64, offset: u64,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
let n = self.0.write_vectored_at(bufs, offset)?; let n = self.0.write_vectored_at(bufs, offset)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }
async fn seek(&mut self, pos: std::io::SeekFrom) -> Result<u64, Error> { async fn seek(&self, pos: std::io::SeekFrom) -> Result<u64, Error> {
Ok(self.0.seek(pos)?) Ok(self.0.seek(pos)?)
} }
async fn peek(&mut self, buf: &mut [u8]) -> Result<u64, Error> { async fn peek(&self, buf: &mut [u8]) -> Result<u64, Error> {
let n = self.0.peek(buf)?; let n = self.0.peek(buf)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }
async fn num_ready_bytes(&self) -> Result<u64, Error> { fn num_ready_bytes(&self) -> Result<u64, Error> {
Ok(self.0.num_ready_bytes()?) Ok(self.0.num_ready_bytes()?)
} }
fn isatty(&mut self) -> bool { fn isatty(&self) -> bool {
self.0.is_terminal() self.0.is_terminal()
} }
} }

10
crates/wasi-common/cap-std-sync/src/lib.rs
View File

@@ -94,15 +94,15 @@ impl WasiCtxBuilder {
} }
Ok(self) Ok(self)
} }
pub fn stdin(mut self, f: Box<dyn WasiFile>) -> Self { pub fn stdin(self, f: Box<dyn WasiFile>) -> Self {
self.0.set_stdin(f); self.0.set_stdin(f);
self self
} }
pub fn stdout(mut self, f: Box<dyn WasiFile>) -> Self { pub fn stdout(self, f: Box<dyn WasiFile>) -> Self {
self.0.set_stdout(f); self.0.set_stdout(f);
self self
} }
pub fn stderr(mut self, f: Box<dyn WasiFile>) -> Self { pub fn stderr(self, f: Box<dyn WasiFile>) -> Self {
self.0.set_stderr(f); self.0.set_stderr(f);
self self
} }
@@ -118,12 +118,12 @@ impl WasiCtxBuilder {
pub fn inherit_stdio(self) -> Self { pub fn inherit_stdio(self) -> Self {
self.inherit_stdin().inherit_stdout().inherit_stderr() self.inherit_stdin().inherit_stdout().inherit_stderr()
} }
pub fn preopened_dir(mut self, dir: Dir, guest_path: impl AsRef<Path>) -> Result<Self, Error> { pub fn preopened_dir(self, dir: Dir, guest_path: impl AsRef<Path>) -> Result<Self, Error> {
let dir = Box::new(crate::dir::Dir::from_cap_std(dir)); let dir = Box::new(crate::dir::Dir::from_cap_std(dir));
self.0.push_preopened_dir(dir, guest_path)?; self.0.push_preopened_dir(dir, guest_path)?;
Ok(self) Ok(self)
} }
pub fn preopened_socket(mut self, fd: u32, socket: impl Into<Socket>) -> Result<Self, Error> { pub fn preopened_socket(self, fd: u32, socket: impl Into<Socket>) -> Result<Self, Error> {
let socket: Socket = socket.into(); let socket: Socket = socket.into();
let file: Box<dyn WasiFile> = socket.into(); let file: Box<dyn WasiFile> = socket.into();

28
crates/wasi-common/cap-std-sync/src/net.rs
View File

@@ -86,22 +86,21 @@ macro_rules! wasi_listen_write_impl {
fn pollable(&self) -> Option<rustix::fd::BorrowedFd> { fn pollable(&self) -> Option<rustix::fd::BorrowedFd> {
Some(self.0.as_fd()) Some(self.0.as_fd())
} }
#[cfg(windows)] #[cfg(windows)]
fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> { fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> {
Some(self.0.as_raw_handle_or_socket()) Some(self.0.as_raw_handle_or_socket())
} }
async fn sock_accept(&mut self, fdflags: FdFlags) -> Result<Box<dyn WasiFile>, Error> { async fn sock_accept(&self, fdflags: FdFlags) -> Result<Box<dyn WasiFile>, Error> {
let (stream, _) = self.0.accept()?; let (stream, _) = self.0.accept()?;
let mut stream = <$stream>::from_cap_std(stream); let mut stream = <$stream>::from_cap_std(stream);
stream.set_fdflags(fdflags).await?; stream.set_fdflags(fdflags).await?;
Ok(Box::new(stream)) Ok(Box::new(stream))
} }
async fn get_filetype(&mut self) -> Result<FileType, Error> { async fn get_filetype(&self) -> Result<FileType, Error> {
Ok(FileType::SocketStream) Ok(FileType::SocketStream)
} }
#[cfg(unix)] #[cfg(unix)]
async fn get_fdflags(&mut self) -> Result<FdFlags, Error> { async fn get_fdflags(&self) -> Result<FdFlags, Error> {
let fdflags = get_fd_flags(&self.0)?; let fdflags = get_fd_flags(&self.0)?;
Ok(fdflags) Ok(fdflags)
} }
@@ -117,7 +116,7 @@ macro_rules! wasi_listen_write_impl {
} }
Ok(()) Ok(())
} }
async fn num_ready_bytes(&self) -> Result<u64, Error> { fn num_ready_bytes(&self) -> Result<u64, Error> {
Ok(1) Ok(1)
} }
} }
@@ -180,16 +179,15 @@ macro_rules! wasi_stream_write_impl {
fn pollable(&self) -> Option<rustix::fd::BorrowedFd> { fn pollable(&self) -> Option<rustix::fd::BorrowedFd> {
Some(self.0.as_fd()) Some(self.0.as_fd())
} }
#[cfg(windows)] #[cfg(windows)]
fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> { fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> {
Some(self.0.as_raw_handle_or_socket()) Some(self.0.as_raw_handle_or_socket())
} }
async fn get_filetype(&mut self) -> Result<FileType, Error> { async fn get_filetype(&self) -> Result<FileType, Error> {
Ok(FileType::SocketStream) Ok(FileType::SocketStream)
} }
#[cfg(unix)] #[cfg(unix)]
async fn get_fdflags(&mut self) -> Result<FdFlags, Error> { async fn get_fdflags(&self) -> Result<FdFlags, Error> {
let fdflags = get_fd_flags(&self.0)?; let fdflags = get_fd_flags(&self.0)?;
Ok(fdflags) Ok(fdflags)
} }
@@ -206,23 +204,23 @@ macro_rules! wasi_stream_write_impl {
Ok(()) Ok(())
} }
async fn read_vectored<'a>( async fn read_vectored<'a>(
&mut self, &self,
bufs: &mut [io::IoSliceMut<'a>], bufs: &mut [io::IoSliceMut<'a>],
) -> Result<u64, Error> { ) -> Result<u64, Error> {
use std::io::Read; use std::io::Read;
let n = Read::read_vectored(&mut &*self.as_socketlike_view::<$std_ty>(), bufs)?; let n = Read::read_vectored(&mut &*self.as_socketlike_view::<$std_ty>(), bufs)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }
async fn write_vectored<'a>(&mut self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> { async fn write_vectored<'a>(&self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> {
use std::io::Write; use std::io::Write;
let n = Write::write_vectored(&mut &*self.as_socketlike_view::<$std_ty>(), bufs)?; let n = Write::write_vectored(&mut &*self.as_socketlike_view::<$std_ty>(), bufs)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }
async fn peek(&mut self, buf: &mut [u8]) -> Result<u64, Error> { async fn peek(&self, buf: &mut [u8]) -> Result<u64, Error> {
let n = self.0.peek(buf)?; let n = self.0.peek(buf)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }
async fn num_ready_bytes(&self) -> Result<u64, Error> { fn num_ready_bytes(&self) -> Result<u64, Error> {
let val = self.as_socketlike_view::<$std_ty>().num_ready_bytes()?; let val = self.as_socketlike_view::<$std_ty>().num_ready_bytes()?;
Ok(val) Ok(val)
} }
@@ -244,7 +242,7 @@ macro_rules! wasi_stream_write_impl {
} }
async fn sock_recv<'a>( async fn sock_recv<'a>(
&mut self, &self,
ri_data: &mut [std::io::IoSliceMut<'a>], ri_data: &mut [std::io::IoSliceMut<'a>],
ri_flags: RiFlags, ri_flags: RiFlags,
) -> Result<(u64, RoFlags), Error> { ) -> Result<(u64, RoFlags), Error> {
@@ -272,7 +270,7 @@ macro_rules! wasi_stream_write_impl {
} }
async fn sock_send<'a>( async fn sock_send<'a>(
&mut self, &self,
si_data: &[std::io::IoSlice<'a>], si_data: &[std::io::IoSlice<'a>],
si_flags: SiFlags, si_flags: SiFlags,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
@@ -284,7 +282,7 @@ macro_rules! wasi_stream_write_impl {
Ok(n as u64) Ok(n as u64)
} }
async fn sock_shutdown(&mut self, how: SdFlags) -> Result<(), Error> { async fn sock_shutdown(&self, how: SdFlags) -> Result<(), Error> {
let how = if how == SdFlags::RD | SdFlags::WR { let how = if how == SdFlags::RD | SdFlags::WR {
cap_std::net::Shutdown::Both cap_std::net::Shutdown::Both
} else if how == SdFlags::RD { } else if how == SdFlags::RD {

2
crates/wasi-common/cap-std-sync/src/sched/unix.rs
View File

@@ -55,7 +55,7 @@ pub async fn poll_oneoff<'a>(poll: &mut Poll<'a>) -> Result<(), Error> {
let revents = pollfd.revents(); let revents = pollfd.revents();
let (nbytes, rwsub) = match rwsub { let (nbytes, rwsub) = match rwsub {
Subscription::Read(sub) => { Subscription::Read(sub) => {
let ready = sub.file.num_ready_bytes().await?; let ready = sub.file.num_ready_bytes()?;
(std::cmp::max(ready, 1), sub) (std::cmp::max(ready, 1), sub)
} }
Subscription::Write(sub) => (0, sub), Subscription::Write(sub) => (0, sub),

2
crates/wasi-common/cap-std-sync/src/sched/windows.rs
View File

@@ -96,7 +96,7 @@ pub async fn poll_oneoff_<'a>(
} }
} }
for r in immediate_reads { for r in immediate_reads {
match r.file.num_ready_bytes().await { match r.file.num_ready_bytes() {
Ok(ready_bytes) => { Ok(ready_bytes) => {
r.complete(ready_bytes, RwEventFlags::empty()); r.complete(ready_bytes, RwEventFlags::empty());
ready = true; ready = true;

33
crates/wasi-common/cap-std-sync/src/stdio.rs
View File

@@ -31,6 +31,7 @@ impl WasiFile for Stdin {
fn as_any(&self) -> &dyn Any { fn as_any(&self) -> &dyn Any {
self self
} }
#[cfg(unix)] #[cfg(unix)]
fn pollable(&self) -> Option<rustix::fd::BorrowedFd> { fn pollable(&self) -> Option<rustix::fd::BorrowedFd> {
Some(self.0.as_fd()) Some(self.0.as_fd())
@@ -40,32 +41,33 @@ impl WasiFile for Stdin {
fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> { fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> {
Some(self.0.as_raw_handle_or_socket()) Some(self.0.as_raw_handle_or_socket())
} }
async fn get_filetype(&mut self) -> Result<FileType, Error> {
async fn get_filetype(&self) -> Result<FileType, Error> {
if self.isatty() { if self.isatty() {
Ok(FileType::CharacterDevice) Ok(FileType::CharacterDevice)
} else { } else {
Ok(FileType::Unknown) Ok(FileType::Unknown)
} }
} }
async fn read_vectored<'a>(&mut self, bufs: &mut [io::IoSliceMut<'a>]) -> Result<u64, Error> { async fn read_vectored<'a>(&self, bufs: &mut [io::IoSliceMut<'a>]) -> Result<u64, Error> {
let n = (&*self.0.as_filelike_view::<File>()).read_vectored(bufs)?; let n = (&*self.0.as_filelike_view::<File>()).read_vectored(bufs)?;
Ok(n.try_into().map_err(|_| Error::range())?) Ok(n.try_into().map_err(|_| Error::range())?)
} }
async fn read_vectored_at<'a>( async fn read_vectored_at<'a>(
&mut self, &self,
_bufs: &mut [io::IoSliceMut<'a>], _bufs: &mut [io::IoSliceMut<'a>],
_offset: u64, _offset: u64,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
Err(Error::seek_pipe()) Err(Error::seek_pipe())
} }
async fn seek(&mut self, _pos: std::io::SeekFrom) -> Result<u64, Error> { async fn seek(&self, _pos: std::io::SeekFrom) -> Result<u64, Error> {
Err(Error::seek_pipe()) Err(Error::seek_pipe())
} }
async fn peek(&mut self, _buf: &mut [u8]) -> Result<u64, Error> { async fn peek(&self, _buf: &mut [u8]) -> Result<u64, Error> {
Err(Error::seek_pipe()) Err(Error::seek_pipe())
} }
async fn set_times( async fn set_times(
&mut self, &self,
atime: Option<wasi_common::SystemTimeSpec>, atime: Option<wasi_common::SystemTimeSpec>,
mtime: Option<wasi_common::SystemTimeSpec>, mtime: Option<wasi_common::SystemTimeSpec>,
) -> Result<(), Error> { ) -> Result<(), Error> {
@@ -73,10 +75,10 @@ impl WasiFile for Stdin {
.set_times(convert_systimespec(atime), convert_systimespec(mtime))?; .set_times(convert_systimespec(atime), convert_systimespec(mtime))?;
Ok(()) Ok(())
} }
async fn num_ready_bytes(&self) -> Result<u64, Error> { fn num_ready_bytes(&self) -> Result<u64, Error> {
Ok(self.0.num_ready_bytes()?) Ok(self.0.num_ready_bytes()?)
} }
fn isatty(&mut self) -> bool { fn isatty(&self) -> bool {
self.0.is_terminal() self.0.is_terminal()
} }
} }
@@ -111,39 +113,38 @@ macro_rules! wasi_file_write_impl {
fn pollable(&self) -> Option<rustix::fd::BorrowedFd> { fn pollable(&self) -> Option<rustix::fd::BorrowedFd> {
Some(self.0.as_fd()) Some(self.0.as_fd())
} }
#[cfg(windows)] #[cfg(windows)]
fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> { fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> {
Some(self.0.as_raw_handle_or_socket()) Some(self.0.as_raw_handle_or_socket())
} }
async fn get_filetype(&mut self) -> Result<FileType, Error> { async fn get_filetype(&self) -> Result<FileType, Error> {
if self.isatty() { if self.isatty() {
Ok(FileType::CharacterDevice) Ok(FileType::CharacterDevice)
} else { } else {
Ok(FileType::Unknown) Ok(FileType::Unknown)
} }
} }
async fn get_fdflags(&mut self) -> Result<FdFlags, Error> { async fn get_fdflags(&self) -> Result<FdFlags, Error> {
Ok(FdFlags::APPEND) Ok(FdFlags::APPEND)
} }
async fn write_vectored<'a>(&mut self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> { async fn write_vectored<'a>(&self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> {
let n = (&*self.0.as_filelike_view::<File>()).write_vectored(bufs)?; let n = (&*self.0.as_filelike_view::<File>()).write_vectored(bufs)?;
Ok(n.try_into().map_err(|_| { Ok(n.try_into().map_err(|_| {
Error::range().context("converting write_vectored total length") Error::range().context("converting write_vectored total length")
})?) })?)
} }
async fn write_vectored_at<'a>( async fn write_vectored_at<'a>(
&mut self, &self,
_bufs: &[io::IoSlice<'a>], _bufs: &[io::IoSlice<'a>],
_offset: u64, _offset: u64,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
Err(Error::seek_pipe()) Err(Error::seek_pipe())
} }
async fn seek(&mut self, _pos: std::io::SeekFrom) -> Result<u64, Error> { async fn seek(&self, _pos: std::io::SeekFrom) -> Result<u64, Error> {
Err(Error::seek_pipe()) Err(Error::seek_pipe())
} }
async fn set_times( async fn set_times(
&mut self, &self,
atime: Option<wasi_common::SystemTimeSpec>, atime: Option<wasi_common::SystemTimeSpec>,
mtime: Option<wasi_common::SystemTimeSpec>, mtime: Option<wasi_common::SystemTimeSpec>,
) -> Result<(), Error> { ) -> Result<(), Error> {
@@ -151,7 +152,7 @@ macro_rules! wasi_file_write_impl {
.set_times(convert_systimespec(atime), convert_systimespec(mtime))?; .set_times(convert_systimespec(atime), convert_systimespec(mtime))?;
Ok(()) Ok(())
} }
fn isatty(&mut self) -> bool { fn isatty(&self) -> bool {
self.0.is_terminal() self.0.is_terminal()
} }
} }

78
crates/wasi-common/src/ctx.rs
View File

@@ -2,16 +2,29 @@ use crate::clocks::WasiClocks;
use crate::dir::{DirCaps, DirEntry, WasiDir}; use crate::dir::{DirCaps, DirEntry, WasiDir};
use crate::file::{FileCaps, FileEntry, WasiFile}; use crate::file::{FileCaps, FileEntry, WasiFile};
use crate::sched::WasiSched; use crate::sched::WasiSched;
use crate::string_array::{StringArray, StringArrayError}; use crate::string_array::StringArray;
use crate::table::Table; use crate::table::Table;
use crate::Error; use crate::{Error, StringArrayError};
use cap_rand::RngCore; use cap_rand::RngCore;
use std::ops::Deref;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
pub struct WasiCtx { /// An `Arc`-wrapper around the wasi-common context to allow mutable access to
/// the file descriptor table. This wrapper is only necessary due to the
/// signature of `fd_fdstat_set_flags`; if that changes, there are a variety of
/// improvements that can be made (TODO:
/// https://github.com/bytecodealliance/wasmtime/issues/5643).
#[derive(Clone)]
pub struct WasiCtx(Arc<WasiCtxInner>);
pub struct WasiCtxInner {
pub args: StringArray, pub args: StringArray,
pub env: StringArray, pub env: StringArray,
pub random: Box<dyn RngCore + Send + Sync>, // TODO: this mutex should not be necessary, it forces threads to serialize
// their access to randomness unnecessarily
// (https://github.com/bytecodealliance/wasmtime/issues/5660).
pub random: Mutex<Box<dyn RngCore + Send + Sync>>,
pub clocks: WasiClocks, pub clocks: WasiClocks,
pub sched: Box<dyn WasiSched>, pub sched: Box<dyn WasiSched>,
pub table: Table, pub table: Table,
@@ -24,31 +37,31 @@ impl WasiCtx {
sched: Box<dyn WasiSched>, sched: Box<dyn WasiSched>,
table: Table, table: Table,
) -> Self { ) -> Self {
let mut s = WasiCtx { let s = WasiCtx(Arc::new(WasiCtxInner {
args: StringArray::new(), args: StringArray::new(),
env: StringArray::new(), env: StringArray::new(),
random, random: Mutex::new(random),
clocks, clocks,
sched, sched,
table, table,
}; }));
s.set_stdin(Box::new(crate::pipe::ReadPipe::new(std::io::empty()))); s.set_stdin(Box::new(crate::pipe::ReadPipe::new(std::io::empty())));
s.set_stdout(Box::new(crate::pipe::WritePipe::new(std::io::sink()))); s.set_stdout(Box::new(crate::pipe::WritePipe::new(std::io::sink())));
s.set_stderr(Box::new(crate::pipe::WritePipe::new(std::io::sink()))); s.set_stderr(Box::new(crate::pipe::WritePipe::new(std::io::sink())));
s s
} }
pub fn insert_file(&mut self, fd: u32, file: Box<dyn WasiFile>, caps: FileCaps) { pub fn insert_file(&self, fd: u32, file: Box<dyn WasiFile>, caps: FileCaps) {
self.table() self.table()
.insert_at(fd, Box::new(FileEntry::new(caps, file))); .insert_at(fd, Arc::new(FileEntry::new(caps, file)));
} }
pub fn push_file(&mut self, file: Box<dyn WasiFile>, caps: FileCaps) -> Result<u32, Error> { pub fn push_file(&self, file: Box<dyn WasiFile>, caps: FileCaps) -> Result<u32, Error> {
self.table().push(Box::new(FileEntry::new(caps, file))) self.table().push(Arc::new(FileEntry::new(caps, file)))
} }
pub fn insert_dir( pub fn insert_dir(
&mut self, &self,
fd: u32, fd: u32,
dir: Box<dyn WasiDir>, dir: Box<dyn WasiDir>,
caps: DirCaps, caps: DirCaps,
@@ -57,45 +70,55 @@ impl WasiCtx {
) { ) {
self.table().insert_at( self.table().insert_at(
fd, fd,
Box::new(DirEntry::new(caps, file_caps, Some(path), dir)), Arc::new(DirEntry::new(caps, file_caps, Some(path), dir)),
); );
} }
pub fn push_dir( pub fn push_dir(
&mut self, &self,
dir: Box<dyn WasiDir>, dir: Box<dyn WasiDir>,
caps: DirCaps, caps: DirCaps,
file_caps: FileCaps, file_caps: FileCaps,
path: PathBuf, path: PathBuf,
) -> Result<u32, Error> { ) -> Result<u32, Error> {
self.table() self.table()
.push(Box::new(DirEntry::new(caps, file_caps, Some(path), dir))) .push(Arc::new(DirEntry::new(caps, file_caps, Some(path), dir)))
} }
pub fn table(&mut self) -> &mut Table { pub fn table(&self) -> &Table {
&mut self.table &self.table
}
pub fn table_mut(&mut self) -> Option<&mut Table> {
Arc::get_mut(&mut self.0).map(|c| &mut c.table)
} }
pub fn push_arg(&mut self, arg: &str) -> Result<(), StringArrayError> { pub fn push_arg(&mut self, arg: &str) -> Result<(), StringArrayError> {
self.args.push(arg.to_owned()) let s = Arc::get_mut(&mut self.0).expect(
"`push_arg` should only be used during initialization before the context is cloned",
);
s.args.push(arg.to_owned())
} }
pub fn push_env(&mut self, var: &str, value: &str) -> Result<(), StringArrayError> { pub fn push_env(&mut self, var: &str, value: &str) -> Result<(), StringArrayError> {
self.env.push(format!("{}={}", var, value))?; let s = Arc::get_mut(&mut self.0).expect(
"`push_env` should only be used during initialization before the context is cloned",
);
s.env.push(format!("{}={}", var, value))?;
Ok(()) Ok(())
} }
pub fn set_stdin(&mut self, mut f: Box<dyn WasiFile>) { pub fn set_stdin(&self, mut f: Box<dyn WasiFile>) {
let rights = Self::stdio_rights(&mut *f); let rights = Self::stdio_rights(&mut *f);
self.insert_file(0, f, rights); self.insert_file(0, f, rights);
} }
pub fn set_stdout(&mut self, mut f: Box<dyn WasiFile>) { pub fn set_stdout(&self, mut f: Box<dyn WasiFile>) {
let rights = Self::stdio_rights(&mut *f); let rights = Self::stdio_rights(&mut *f);
self.insert_file(1, f, rights); self.insert_file(1, f, rights);
} }
pub fn set_stderr(&mut self, mut f: Box<dyn WasiFile>) { pub fn set_stderr(&self, mut f: Box<dyn WasiFile>) {
let rights = Self::stdio_rights(&mut *f); let rights = Self::stdio_rights(&mut *f);
self.insert_file(2, f, rights); self.insert_file(2, f, rights);
} }
@@ -114,13 +137,13 @@ impl WasiCtx {
} }
pub fn push_preopened_dir( pub fn push_preopened_dir(
&mut self, &self,
dir: Box<dyn WasiDir>, dir: Box<dyn WasiDir>,
path: impl AsRef<Path>, path: impl AsRef<Path>,
) -> Result<(), Error> { ) -> Result<(), Error> {
let caps = DirCaps::all(); let caps = DirCaps::all();
let file_caps = FileCaps::all(); let file_caps = FileCaps::all();
self.table().push(Box::new(DirEntry::new( self.table().push(Arc::new(DirEntry::new(
caps, caps,
file_caps, file_caps,
Some(path.as_ref().to_owned()), Some(path.as_ref().to_owned()),
@@ -129,3 +152,10 @@ impl WasiCtx {
Ok(()) Ok(())
} }
} }
impl Deref for WasiCtx {
type Target = WasiCtxInner;
fn deref(&self) -> &Self::Target {
&self.0
}
}

93
crates/wasi-common/src/dir.rs
View File

@@ -3,6 +3,7 @@ use crate::{Error, ErrorExt, SystemTimeSpec};
use bitflags::bitflags; use bitflags::bitflags;
use std::any::Any; use std::any::Any;
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::{Arc, RwLock};
#[wiggle::async_trait] #[wiggle::async_trait]
pub trait WasiDir: Send + Sync { pub trait WasiDir: Send + Sync {
@@ -98,67 +99,50 @@ pub trait WasiDir: Send + Sync {
} }
pub(crate) struct DirEntry { pub(crate) struct DirEntry {
caps: DirCaps, caps: RwLock<DirFdStat>,
file_caps: FileCaps,
preopen_path: Option<PathBuf>, // precondition: PathBuf is valid unicode preopen_path: Option<PathBuf>, // precondition: PathBuf is valid unicode
dir: Box<dyn WasiDir>, dir: Box<dyn WasiDir>,
} }
impl DirEntry { impl DirEntry {
pub fn new( pub fn new(
caps: DirCaps, dir_caps: DirCaps,
file_caps: FileCaps, file_caps: FileCaps,
preopen_path: Option<PathBuf>, preopen_path: Option<PathBuf>,
dir: Box<dyn WasiDir>, dir: Box<dyn WasiDir>,
) -> Self { ) -> Self {
DirEntry { DirEntry {
caps, caps: RwLock::new(DirFdStat {
file_caps, dir_caps,
file_caps,
}),
preopen_path, preopen_path,
dir, dir,
} }
} }
pub fn capable_of_dir(&self, caps: DirCaps) -> Result<(), Error> { pub fn capable_of_dir(&self, caps: DirCaps) -> Result<(), Error> {
if self.caps.contains(caps) { let fdstat = self.caps.read().unwrap();
Ok(()) fdstat.capable_of_dir(caps)
} else {
let missing = caps & !self.caps;
let err = if missing.intersects(DirCaps::READDIR) {
Error::not_dir()
} else {
Error::perm()
};
Err(err.context(format!("desired rights {:?}, has {:?}", caps, self.caps)))
}
} }
pub fn capable_of_file(&self, caps: FileCaps) -> Result<(), Error> {
if self.file_caps.contains(caps) { pub fn drop_caps_to(&self, dir_caps: DirCaps, file_caps: FileCaps) -> Result<(), Error> {
Ok(()) let mut fdstat = self.caps.write().unwrap();
} else { fdstat.capable_of_dir(dir_caps)?;
Err(Error::perm().context(format!( fdstat.capable_of_file(file_caps)?;
"desired rights {:?}, has {:?}", *fdstat = DirFdStat {
caps, self.file_caps dir_caps,
))) file_caps,
} };
}
pub fn drop_caps_to(&mut self, caps: DirCaps, file_caps: FileCaps) -> Result<(), Error> {
self.capable_of_dir(caps)?;
self.capable_of_file(file_caps)?;
self.caps = caps;
self.file_caps = file_caps;
Ok(()) Ok(())
} }
pub fn child_dir_caps(&self, desired_caps: DirCaps) -> DirCaps { pub fn child_dir_caps(&self, desired_caps: DirCaps) -> DirCaps {
self.caps & desired_caps self.caps.read().unwrap().dir_caps & desired_caps
} }
pub fn child_file_caps(&self, desired_caps: FileCaps) -> FileCaps { pub fn child_file_caps(&self, desired_caps: FileCaps) -> FileCaps {
self.file_caps & desired_caps self.caps.read().unwrap().file_caps & desired_caps
} }
pub fn get_dir_fdstat(&self) -> DirFdStat { pub fn get_dir_fdstat(&self) -> DirFdStat {
DirFdStat { self.caps.read().unwrap().clone()
dir_caps: self.caps,
file_caps: self.file_caps,
}
} }
pub fn preopen_path(&self) -> &Option<PathBuf> { pub fn preopen_path(&self) -> &Option<PathBuf> {
&self.preopen_path &self.preopen_path
@@ -203,18 +187,47 @@ pub struct DirFdStat {
pub dir_caps: DirCaps, pub dir_caps: DirCaps,
} }
impl DirFdStat {
pub fn capable_of_dir(&self, caps: DirCaps) -> Result<(), Error> {
if self.dir_caps.contains(caps) {
Ok(())
} else {
let missing = caps & !self.dir_caps;
let err = if missing.intersects(DirCaps::READDIR) {
Error::not_dir()
} else {
Error::perm()
};
Err(err.context(format!(
"desired rights {:?}, has {:?}",
caps, self.dir_caps
)))
}
}
pub fn capable_of_file(&self, caps: FileCaps) -> Result<(), Error> {
if self.file_caps.contains(caps) {
Ok(())
} else {
Err(Error::perm().context(format!(
"desired rights {:?}, has {:?}",
caps, self.file_caps
)))
}
}
}
pub(crate) trait TableDirExt { pub(crate) trait TableDirExt {
fn get_dir(&self, fd: u32) -> Result<&DirEntry, Error>; fn get_dir(&self, fd: u32) -> Result<Arc<DirEntry>, Error>;
fn is_preopen(&self, fd: u32) -> bool; fn is_preopen(&self, fd: u32) -> bool;
} }
impl TableDirExt for crate::table::Table { impl TableDirExt for crate::table::Table {
fn get_dir(&self, fd: u32) -> Result<&DirEntry, Error> { fn get_dir(&self, fd: u32) -> Result<Arc<DirEntry>, Error> {
self.get(fd) self.get(fd)
} }
fn is_preopen(&self, fd: u32) -> bool { fn is_preopen(&self, fd: u32) -> bool {
if self.is::<DirEntry>(fd) { if self.is::<DirEntry>(fd) {
let dir_entry: &DirEntry = self.get(fd).unwrap(); let dir_entry: Arc<DirEntry> = self.get(fd).unwrap();
dir_entry.preopen_path.is_some() dir_entry.preopen_path.is_some()
} else { } else {
false false

71
crates/wasi-common/src/file.rs
View File

@@ -1,11 +1,12 @@
use crate::{Error, ErrorExt, SystemTimeSpec}; use crate::{Error, ErrorExt, SystemTimeSpec};
use bitflags::bitflags; use bitflags::bitflags;
use std::any::Any; use std::any::Any;
use std::sync::{Arc, RwLock};
#[wiggle::async_trait] #[wiggle::async_trait]
pub trait WasiFile: Send + Sync { pub trait WasiFile: Send + Sync {
fn as_any(&self) -> &dyn Any; fn as_any(&self) -> &dyn Any;
async fn get_filetype(&mut self) -> Result<FileType, Error>; async fn get_filetype(&self) -> Result<FileType, Error>;
#[cfg(unix)] #[cfg(unix)]
fn pollable(&self) -> Option<rustix::fd::BorrowedFd> { fn pollable(&self) -> Option<rustix::fd::BorrowedFd> {
@@ -17,16 +18,16 @@ pub trait WasiFile: Send + Sync {
None None
} }
fn isatty(&mut self) -> bool { fn isatty(&self) -> bool {
false false
} }
async fn sock_accept(&mut self, _fdflags: FdFlags) -> Result<Box<dyn WasiFile>, Error> { async fn sock_accept(&self, _fdflags: FdFlags) -> Result<Box<dyn WasiFile>, Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn sock_recv<'a>( async fn sock_recv<'a>(
&mut self, &self,
_ri_data: &mut [std::io::IoSliceMut<'a>], _ri_data: &mut [std::io::IoSliceMut<'a>],
_ri_flags: RiFlags, _ri_flags: RiFlags,
) -> Result<(u64, RoFlags), Error> { ) -> Result<(u64, RoFlags), Error> {
@@ -34,26 +35,26 @@ pub trait WasiFile: Send + Sync {
} }
async fn sock_send<'a>( async fn sock_send<'a>(
&mut self, &self,
_si_data: &[std::io::IoSlice<'a>], _si_data: &[std::io::IoSlice<'a>],
_si_flags: SiFlags, _si_flags: SiFlags,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn sock_shutdown(&mut self, _how: SdFlags) -> Result<(), Error> { async fn sock_shutdown(&self, _how: SdFlags) -> Result<(), Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn datasync(&mut self) -> Result<(), Error> { async fn datasync(&self) -> Result<(), Error> {
Ok(()) Ok(())
} }
async fn sync(&mut self) -> Result<(), Error> { async fn sync(&self) -> Result<(), Error> {
Ok(()) Ok(())
} }
async fn get_fdflags(&mut self) -> Result<FdFlags, Error> { async fn get_fdflags(&self) -> Result<FdFlags, Error> {
Ok(FdFlags::empty()) Ok(FdFlags::empty())
} }
@@ -61,7 +62,7 @@ pub trait WasiFile: Send + Sync {
Err(Error::badf()) Err(Error::badf())
} }
async fn get_filestat(&mut self) -> Result<Filestat, Error> { async fn get_filestat(&self) -> Result<Filestat, Error> {
Ok(Filestat { Ok(Filestat {
device_id: 0, device_id: 0,
inode: 0, inode: 0,
@@ -74,62 +75,59 @@ pub trait WasiFile: Send + Sync {
}) })
} }
async fn set_filestat_size(&mut self, _size: u64) -> Result<(), Error> { async fn set_filestat_size(&self, _size: u64) -> Result<(), Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn advise(&mut self, _offset: u64, _len: u64, _advice: Advice) -> Result<(), Error> { async fn advise(&self, _offset: u64, _len: u64, _advice: Advice) -> Result<(), Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn allocate(&mut self, _offset: u64, _len: u64) -> Result<(), Error> { async fn allocate(&self, _offset: u64, _len: u64) -> Result<(), Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn set_times( async fn set_times(
&mut self, &self,
_atime: Option<SystemTimeSpec>, _atime: Option<SystemTimeSpec>,
_mtime: Option<SystemTimeSpec>, _mtime: Option<SystemTimeSpec>,
) -> Result<(), Error> { ) -> Result<(), Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn read_vectored<'a>( async fn read_vectored<'a>(&self, _bufs: &mut [std::io::IoSliceMut<'a>]) -> Result<u64, Error> {
&mut self,
_bufs: &mut [std::io::IoSliceMut<'a>],
) -> Result<u64, Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn read_vectored_at<'a>( async fn read_vectored_at<'a>(
&mut self, &self,
_bufs: &mut [std::io::IoSliceMut<'a>], _bufs: &mut [std::io::IoSliceMut<'a>],
_offset: u64, _offset: u64,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn write_vectored<'a>(&mut self, _bufs: &[std::io::IoSlice<'a>]) -> Result<u64, Error> { async fn write_vectored<'a>(&self, _bufs: &[std::io::IoSlice<'a>]) -> Result<u64, Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn write_vectored_at<'a>( async fn write_vectored_at<'a>(
&mut self, &self,
_bufs: &[std::io::IoSlice<'a>], _bufs: &[std::io::IoSlice<'a>],
_offset: u64, _offset: u64,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn seek(&mut self, _pos: std::io::SeekFrom) -> Result<u64, Error> { async fn seek(&self, _pos: std::io::SeekFrom) -> Result<u64, Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn peek(&mut self, _buf: &mut [u8]) -> Result<u64, Error> { async fn peek(&self, _buf: &mut [u8]) -> Result<u64, Error> {
Err(Error::badf()) Err(Error::badf())
} }
async fn num_ready_bytes(&self) -> Result<u64, Error> { fn num_ready_bytes(&self) -> Result<u64, Error> {
Ok(0) Ok(0)
} }
@@ -212,11 +210,11 @@ pub struct Filestat {
} }
pub(crate) trait TableFileExt { pub(crate) trait TableFileExt {
fn get_file(&self, fd: u32) -> Result<&FileEntry, Error>; fn get_file(&self, fd: u32) -> Result<Arc<FileEntry>, Error>;
fn get_file_mut(&mut self, fd: u32) -> Result<&mut FileEntry, Error>; fn get_file_mut(&mut self, fd: u32) -> Result<&mut FileEntry, Error>;
} }
impl TableFileExt for crate::table::Table { impl TableFileExt for crate::table::Table {
fn get_file(&self, fd: u32) -> Result<&FileEntry, Error> { fn get_file(&self, fd: u32) -> Result<Arc<FileEntry>, Error> {
self.get(fd) self.get(fd)
} }
fn get_file_mut(&mut self, fd: u32) -> Result<&mut FileEntry, Error> { fn get_file_mut(&mut self, fd: u32) -> Result<&mut FileEntry, Error> {
@@ -225,20 +223,23 @@ impl TableFileExt for crate::table::Table {
} }
pub(crate) struct FileEntry { pub(crate) struct FileEntry {
caps: FileCaps, caps: RwLock<FileCaps>,
file: Box<dyn WasiFile>, file: Box<dyn WasiFile>,
} }
impl FileEntry { impl FileEntry {
pub fn new(caps: FileCaps, file: Box<dyn WasiFile>) -> Self { pub fn new(caps: FileCaps, file: Box<dyn WasiFile>) -> Self {
FileEntry { caps, file } FileEntry {
caps: RwLock::new(caps),
file,
}
} }
pub fn capable_of(&self, caps: FileCaps) -> Result<(), Error> { pub fn capable_of(&self, caps: FileCaps) -> Result<(), Error> {
if self.caps.contains(caps) { if self.caps.read().unwrap().contains(caps) {
Ok(()) Ok(())
} else { } else {
let missing = caps & !self.caps; let missing = caps & !(*self.caps.read().unwrap());
let err = if missing.intersects(FileCaps::READ | FileCaps::WRITE) { let err = if missing.intersects(FileCaps::READ | FileCaps::WRITE) {
// `EBADF` is a little surprising here because it's also used // `EBADF` is a little surprising here because it's also used
// for unknown-file-descriptor errors, but it's what POSIX uses // for unknown-file-descriptor errors, but it's what POSIX uses
@@ -251,16 +252,17 @@ impl FileEntry {
} }
} }
pub fn drop_caps_to(&mut self, caps: FileCaps) -> Result<(), Error> { pub fn drop_caps_to(&self, caps: FileCaps) -> Result<(), Error> {
self.capable_of(caps)?; self.capable_of(caps)?;
self.caps = caps; *self.caps.write().unwrap() = caps;
Ok(()) Ok(())
} }
pub async fn get_fdstat(&mut self) -> Result<FdStat, Error> { pub async fn get_fdstat(&self) -> Result<FdStat, Error> {
let caps = self.caps.read().unwrap().clone();
Ok(FdStat { Ok(FdStat {
filetype: self.file.get_filetype().await?, filetype: self.file.get_filetype().await?,
caps: self.caps, caps,
flags: self.file.get_fdflags().await?, flags: self.file.get_fdflags().await?,
}) })
} }
@@ -276,7 +278,6 @@ impl FileEntryExt for FileEntry {
self.capable_of(caps)?; self.capable_of(caps)?;
Ok(&*self.file) Ok(&*self.file)
} }
fn get_cap_mut(&mut self, caps: FileCaps) -> Result<&mut dyn WasiFile, Error> { fn get_cap_mut(&mut self, caps: FileCaps) -> Result<&mut dyn WasiFile, Error> {
self.capable_of(caps)?; self.capable_of(caps)?;
Ok(&mut *self.file) Ok(&mut *self.file)

10
crates/wasi-common/src/pipe.rs
View File

@@ -105,10 +105,10 @@ impl<R: Read + Any + Send + Sync> WasiFile for ReadPipe<R> {
fn as_any(&self) -> &dyn Any { fn as_any(&self) -> &dyn Any {
self self
} }
async fn get_filetype(&mut self) -> Result<FileType, Error> { async fn get_filetype(&self) -> Result<FileType, Error> {
Ok(FileType::Pipe) Ok(FileType::Pipe)
} }
async fn read_vectored<'a>(&mut self, bufs: &mut [io::IoSliceMut<'a>]) -> Result<u64, Error> { async fn read_vectored<'a>(&self, bufs: &mut [io::IoSliceMut<'a>]) -> Result<u64, Error> {
let n = self.borrow().read_vectored(bufs)?; let n = self.borrow().read_vectored(bufs)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }
@@ -189,13 +189,13 @@ impl<W: Write + Any + Send + Sync> WasiFile for WritePipe<W> {
fn as_any(&self) -> &dyn Any { fn as_any(&self) -> &dyn Any {
self self
} }
async fn get_filetype(&mut self) -> Result<FileType, Error> { async fn get_filetype(&self) -> Result<FileType, Error> {
Ok(FileType::Pipe) Ok(FileType::Pipe)
} }
async fn get_fdflags(&mut self) -> Result<FdFlags, Error> { async fn get_fdflags(&self) -> Result<FdFlags, Error> {
Ok(FdFlags::APPEND) Ok(FdFlags::APPEND)
} }
async fn write_vectored<'a>(&mut self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> { async fn write_vectored<'a>(&self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> {
let n = self.borrow().write_vectored(bufs)?; let n = self.borrow().write_vectored(bufs)?;
Ok(n.try_into()?) Ok(n.try_into()?)
} }

34
crates/wasi-common/src/snapshots/preview_0.rs
View File

@@ -528,10 +528,8 @@ impl wasi_unstable::WasiUnstable for WasiCtx {
fd: types::Fd, fd: types::Fd,
iovs: &types::IovecArray<'a>, iovs: &types::IovecArray<'a>,
) -> Result<types::Size, Error> { ) -> Result<types::Size, Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::READ)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::READ)?;
let iovs: Vec<wiggle::GuestPtr<[u8]>> = iovs let iovs: Vec<wiggle::GuestPtr<[u8]>> = iovs
.iter() .iter()
@@ -601,10 +599,8 @@ impl wasi_unstable::WasiUnstable for WasiCtx {
iovs: &types::IovecArray<'a>, iovs: &types::IovecArray<'a>,
offset: types::Filesize, offset: types::Filesize,
) -> Result<types::Size, Error> { ) -> Result<types::Size, Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::READ | FileCaps::SEEK)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::READ | FileCaps::SEEK)?;
let iovs: Vec<wiggle::GuestPtr<[u8]>> = iovs let iovs: Vec<wiggle::GuestPtr<[u8]>> = iovs
.iter() .iter()
@@ -675,10 +671,8 @@ impl wasi_unstable::WasiUnstable for WasiCtx {
fd: types::Fd, fd: types::Fd,
ciovs: &types::CiovecArray<'a>, ciovs: &types::CiovecArray<'a>,
) -> Result<types::Size, Error> { ) -> Result<types::Size, Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::WRITE)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::WRITE)?;
let guest_slices: Vec<wiggle::GuestCow<u8>> = ciovs let guest_slices: Vec<wiggle::GuestCow<u8>> = ciovs
.iter() .iter()
@@ -704,10 +698,8 @@ impl wasi_unstable::WasiUnstable for WasiCtx {
ciovs: &types::CiovecArray<'a>, ciovs: &types::CiovecArray<'a>,
offset: types::Filesize, offset: types::Filesize,
) -> Result<types::Size, Error> { ) -> Result<types::Size, Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::WRITE | FileCaps::SEEK)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::WRITE | FileCaps::SEEK)?;
let guest_slices: Vec<wiggle::GuestCow<u8>> = ciovs let guest_slices: Vec<wiggle::GuestCow<u8>> = ciovs
.iter() .iter()
@@ -953,7 +945,7 @@ impl wasi_unstable::WasiUnstable for WasiCtx {
} }
} }
let table = &mut self.table; let table = &self.table;
let mut sub_fds: HashSet<types::Fd> = HashSet::new(); let mut sub_fds: HashSet<types::Fd> = HashSet::new();
// We need these refmuts to outlive Poll, which will hold the &mut dyn WasiFile inside // We need these refmuts to outlive Poll, which will hold the &mut dyn WasiFile inside
let mut reads: Vec<(u32, Userdata)> = Vec::new(); let mut reads: Vec<(u32, Userdata)> = Vec::new();
@@ -1003,8 +995,8 @@ impl wasi_unstable::WasiUnstable for WasiCtx {
sub_fds.insert(fd); sub_fds.insert(fd);
} }
table table
.get_file_mut(u32::from(fd))? .get_file(u32::from(fd))?
.get_cap_mut(FileCaps::POLL_READWRITE)?; .get_cap(FileCaps::POLL_READWRITE)?;
reads.push((u32::from(fd), sub.userdata.into())); reads.push((u32::from(fd), sub.userdata.into()));
} }
types::SubscriptionU::FdWrite(writesub) => { types::SubscriptionU::FdWrite(writesub) => {
@@ -1016,8 +1008,8 @@ impl wasi_unstable::WasiUnstable for WasiCtx {
sub_fds.insert(fd); sub_fds.insert(fd);
} }
table table
.get_file_mut(u32::from(fd))? .get_file(u32::from(fd))?
.get_cap_mut(FileCaps::POLL_READWRITE)?; .get_cap(FileCaps::POLL_READWRITE)?;
writes.push((u32::from(fd), sub.userdata.into())); writes.push((u32::from(fd), sub.userdata.into()));
} }
} }

192
crates/wasi-common/src/snapshots/preview_1.rs
View File

@@ -14,6 +14,7 @@ use cap_std::time::{Duration, SystemClock};
use std::convert::{TryFrom, TryInto}; use std::convert::{TryFrom, TryInto};
use std::io::{IoSlice, IoSliceMut}; use std::io::{IoSlice, IoSliceMut};
use std::ops::Deref; use std::ops::Deref;
use std::sync::Arc;
use wiggle::GuestPtr; use wiggle::GuestPtr;
pub mod error; pub mod error;
@@ -111,8 +112,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
advice: types::Advice, advice: types::Advice,
) -> Result<(), Error> { ) -> Result<(), Error> {
self.table() self.table()
.get_file_mut(u32::from(fd))? .get_file(u32::from(fd))?
.get_cap_mut(FileCaps::ADVISE)? .get_cap(FileCaps::ADVISE)?
.advise(offset, len, advice.into()) .advise(offset, len, advice.into())
.await?; .await?;
Ok(()) Ok(())
@@ -125,8 +126,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
len: types::Filesize, len: types::Filesize,
) -> Result<(), Error> { ) -> Result<(), Error> {
self.table() self.table()
.get_file_mut(u32::from(fd))? .get_file(u32::from(fd))?
.get_cap_mut(FileCaps::ALLOCATE)? .get_cap(FileCaps::ALLOCATE)?
.allocate(offset, len) .allocate(offset, len)
.await?; .await?;
Ok(()) Ok(())
@@ -142,15 +143,15 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
} }
// fd_close must close either a File or a Dir handle // fd_close must close either a File or a Dir handle
if table.is::<FileEntry>(fd) { if table.is::<FileEntry>(fd) {
let _ = table.delete(fd); let _ = table.delete::<FileEntry>(fd);
} else if table.is::<DirEntry>(fd) { } else if table.is::<DirEntry>(fd) {
// We cannot close preopened directories // We cannot close preopened directories
let dir_entry: &DirEntry = table.get(fd).unwrap(); let dir_entry: Arc<DirEntry> = table.get(fd).unwrap();
if dir_entry.preopen_path().is_some() { if dir_entry.preopen_path().is_some() {
return Err(Error::not_supported().context("cannot close propened directory")); return Err(Error::not_supported().context("cannot close propened directory"));
} }
drop(dir_entry); drop(dir_entry);
let _ = table.delete(fd); let _ = table.delete::<DirEntry>(fd);
} else { } else {
return Err(Error::badf().context("key does not refer to file or directory")); return Err(Error::badf().context("key does not refer to file or directory"));
} }
@@ -160,8 +161,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
async fn fd_datasync(&mut self, fd: types::Fd) -> Result<(), Error> { async fn fd_datasync(&mut self, fd: types::Fd) -> Result<(), Error> {
self.table() self.table()
.get_file_mut(u32::from(fd))? .get_file(u32::from(fd))?
.get_cap_mut(FileCaps::DATASYNC)? .get_cap(FileCaps::DATASYNC)?
.datasync() .datasync()
.await?; .await?;
Ok(()) Ok(())
@@ -171,11 +172,11 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
let table = self.table(); let table = self.table();
let fd = u32::from(fd); let fd = u32::from(fd);
if table.is::<FileEntry>(fd) { if table.is::<FileEntry>(fd) {
let file_entry: &mut FileEntry = table.get_mut(fd)?; let file_entry: Arc<FileEntry> = table.get(fd)?;
let fdstat = file_entry.get_fdstat().await?; let fdstat = file_entry.get_fdstat().await?;
Ok(types::Fdstat::from(&fdstat)) Ok(types::Fdstat::from(&fdstat))
} else if table.is::<DirEntry>(fd) { } else if table.is::<DirEntry>(fd) {
let dir_entry: &DirEntry = table.get(fd)?; let dir_entry: Arc<DirEntry> = table.get(fd)?;
let dir_fdstat = dir_entry.get_dir_fdstat(); let dir_fdstat = dir_entry.get_dir_fdstat();
Ok(types::Fdstat::from(&dir_fdstat)) Ok(types::Fdstat::from(&dir_fdstat))
} else { } else {
@@ -188,11 +189,16 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
fd: types::Fd, fd: types::Fd,
flags: types::Fdflags, flags: types::Fdflags,
) -> Result<(), Error> { ) -> Result<(), Error> {
self.table() if let Some(table) = self.table_mut() {
.get_file_mut(u32::from(fd))? table
.get_cap_mut(FileCaps::FDSTAT_SET_FLAGS)? .get_file_mut(u32::from(fd))?
.set_fdflags(FdFlags::from(flags)) .get_cap_mut(FileCaps::FDSTAT_SET_FLAGS)?
.await .set_fdflags(FdFlags::from(flags))
.await
} else {
log::warn!("`fd_fdstat_set_flags` does not work with wasi-threads enabled; see https://github.com/bytecodealliance/wasmtime/issues/5643");
Err(Error::not_supported())
}
} }
async fn fd_fdstat_set_rights( async fn fd_fdstat_set_rights(
@@ -204,11 +210,11 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
let table = self.table(); let table = self.table();
let fd = u32::from(fd); let fd = u32::from(fd);
if table.is::<FileEntry>(fd) { if table.is::<FileEntry>(fd) {
let file_entry: &mut FileEntry = table.get_mut(fd)?; let file_entry: Arc<FileEntry> = table.get(fd)?;
let file_caps = FileCaps::from(&fs_rights_base); let file_caps = FileCaps::from(&fs_rights_base);
file_entry.drop_caps_to(file_caps) file_entry.drop_caps_to(file_caps)
} else if table.is::<DirEntry>(fd) { } else if table.is::<DirEntry>(fd) {
let dir_entry: &mut DirEntry = table.get_mut(fd)?; let dir_entry: Arc<DirEntry> = table.get(fd)?;
let dir_caps = DirCaps::from(&fs_rights_base); let dir_caps = DirCaps::from(&fs_rights_base);
let file_caps = FileCaps::from(&fs_rights_inheriting); let file_caps = FileCaps::from(&fs_rights_inheriting);
dir_entry.drop_caps_to(dir_caps, file_caps) dir_entry.drop_caps_to(dir_caps, file_caps)
@@ -222,8 +228,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
let fd = u32::from(fd); let fd = u32::from(fd);
if table.is::<FileEntry>(fd) { if table.is::<FileEntry>(fd) {
let filestat = table let filestat = table
.get_file_mut(fd)? .get_file(fd)?
.get_cap_mut(FileCaps::FILESTAT_GET)? .get_cap(FileCaps::FILESTAT_GET)?
.get_filestat() .get_filestat()
.await?; .await?;
Ok(filestat.into()) Ok(filestat.into())
@@ -245,8 +251,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
size: types::Filesize, size: types::Filesize,
) -> Result<(), Error> { ) -> Result<(), Error> {
self.table() self.table()
.get_file_mut(u32::from(fd))? .get_file(u32::from(fd))?
.get_cap_mut(FileCaps::FILESTAT_SET_SIZE)? .get_cap(FileCaps::FILESTAT_SET_SIZE)?
.set_filestat_size(size) .set_filestat_size(size)
.await?; .await?;
Ok(()) Ok(())
@@ -272,9 +278,9 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
if table.is::<FileEntry>(fd) { if table.is::<FileEntry>(fd) {
table table
.get_file_mut(fd) .get_file(fd)
.expect("checked that entry is file") .expect("checked that entry is file")
.get_cap_mut(FileCaps::FILESTAT_SET_TIMES)? .get_cap(FileCaps::FILESTAT_SET_TIMES)?
.set_times(atim, mtim) .set_times(atim, mtim)
.await .await
} else if table.is::<DirEntry>(fd) { } else if table.is::<DirEntry>(fd) {
@@ -294,10 +300,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
fd: types::Fd, fd: types::Fd,
iovs: &types::IovecArray<'a>, iovs: &types::IovecArray<'a>,
) -> Result<types::Size, Error> { ) -> Result<types::Size, Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::READ)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::READ)?;
let iovs: Vec<wiggle::GuestPtr<[u8]>> = iovs let iovs: Vec<wiggle::GuestPtr<[u8]>> = iovs
.iter() .iter()
@@ -367,10 +371,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
iovs: &types::IovecArray<'a>, iovs: &types::IovecArray<'a>,
offset: types::Filesize, offset: types::Filesize,
) -> Result<types::Size, Error> { ) -> Result<types::Size, Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::READ | FileCaps::SEEK)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::READ | FileCaps::SEEK)?;
let iovs: Vec<wiggle::GuestPtr<[u8]>> = iovs let iovs: Vec<wiggle::GuestPtr<[u8]>> = iovs
.iter() .iter()
@@ -441,10 +443,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
fd: types::Fd, fd: types::Fd,
ciovs: &types::CiovecArray<'a>, ciovs: &types::CiovecArray<'a>,
) -> Result<types::Size, Error> { ) -> Result<types::Size, Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::WRITE)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::WRITE)?;
let guest_slices: Vec<wiggle::GuestCow<u8>> = ciovs let guest_slices: Vec<wiggle::GuestCow<u8>> = ciovs
.iter() .iter()
@@ -470,10 +470,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
ciovs: &types::CiovecArray<'a>, ciovs: &types::CiovecArray<'a>,
offset: types::Filesize, offset: types::Filesize,
) -> Result<types::Size, Error> { ) -> Result<types::Size, Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::WRITE | FileCaps::SEEK)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::WRITE | FileCaps::SEEK)?;
let guest_slices: Vec<wiggle::GuestCow<u8>> = ciovs let guest_slices: Vec<wiggle::GuestCow<u8>> = ciovs
.iter() .iter()
@@ -495,7 +493,7 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
async fn fd_prestat_get(&mut self, fd: types::Fd) -> Result<types::Prestat, Error> { async fn fd_prestat_get(&mut self, fd: types::Fd) -> Result<types::Prestat, Error> {
let table = self.table(); let table = self.table();
let dir_entry: &DirEntry = table.get(u32::from(fd)).map_err(|_| Error::badf())?; let dir_entry: Arc<DirEntry> = table.get(u32::from(fd)).map_err(|_| Error::badf())?;
if let Some(ref preopen) = dir_entry.preopen_path() { if let Some(ref preopen) = dir_entry.preopen_path() {
let path_str = preopen.to_str().ok_or_else(|| Error::not_supported())?; let path_str = preopen.to_str().ok_or_else(|| Error::not_supported())?;
let pr_name_len = u32::try_from(path_str.as_bytes().len())?; let pr_name_len = u32::try_from(path_str.as_bytes().len())?;
@@ -512,7 +510,7 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
path_max_len: types::Size, path_max_len: types::Size,
) -> Result<(), Error> { ) -> Result<(), Error> {
let table = self.table(); let table = self.table();
let dir_entry: &DirEntry = table.get(u32::from(fd)).map_err(|_| Error::not_dir())?; let dir_entry: Arc<DirEntry> = table.get(u32::from(fd)).map_err(|_| Error::not_dir())?;
if let Some(ref preopen) = dir_entry.preopen_path() { if let Some(ref preopen) = dir_entry.preopen_path() {
let path_bytes = preopen let path_bytes = preopen
.to_str() .to_str()
@@ -538,11 +536,7 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
if table.is_preopen(from) || table.is_preopen(to) { if table.is_preopen(from) || table.is_preopen(to) {
return Err(Error::not_supported().context("cannot renumber a preopen")); return Err(Error::not_supported().context("cannot renumber a preopen"));
} }
let from_entry = table table.renumber(from, to)
.delete(from)
.expect("we checked that table contains from");
table.insert_at(to, from_entry);
Ok(())
} }
async fn fd_seek( async fn fd_seek(
@@ -566,8 +560,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
}; };
let newoffset = self let newoffset = self
.table() .table()
.get_file_mut(u32::from(fd))? .get_file(u32::from(fd))?
.get_cap_mut(required_caps)? .get_cap(required_caps)?
.seek(whence) .seek(whence)
.await?; .await?;
Ok(newoffset) Ok(newoffset)
@@ -575,8 +569,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
async fn fd_sync(&mut self, fd: types::Fd) -> Result<(), Error> { async fn fd_sync(&mut self, fd: types::Fd) -> Result<(), Error> {
self.table() self.table()
.get_file_mut(u32::from(fd))? .get_file(u32::from(fd))?
.get_cap_mut(FileCaps::SYNC)? .get_cap(FileCaps::SYNC)?
.sync() .sync()
.await?; .await?;
Ok(()) Ok(())
@@ -586,8 +580,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
// XXX should this be stream_position? // XXX should this be stream_position?
let offset = self let offset = self
.table() .table()
.get_file_mut(u32::from(fd))? .get_file(u32::from(fd))?
.get_cap_mut(FileCaps::TELL)? .get_cap(FileCaps::TELL)?
.seek(std::io::SeekFrom::Current(0)) .seek(std::io::SeekFrom::Current(0))
.await?; .await?;
Ok(offset) Ok(offset)
@@ -714,12 +708,10 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
target_path: &GuestPtr<'a, str>, target_path: &GuestPtr<'a, str>,
) -> Result<(), Error> { ) -> Result<(), Error> {
let table = self.table(); let table = self.table();
let src_dir = table let src_dir = table.get_dir(u32::from(src_fd))?;
.get_dir(u32::from(src_fd))? let src_dir = src_dir.get_cap(DirCaps::LINK_SOURCE)?;
.get_cap(DirCaps::LINK_SOURCE)?; let target_dir = table.get_dir(u32::from(target_fd))?;
let target_dir = table let target_dir = target_dir.get_cap(DirCaps::LINK_TARGET)?;
.get_dir(u32::from(target_fd))?
.get_cap(DirCaps::LINK_TARGET)?;
let symlink_follow = src_flags.contains(types::Lookupflags::SYMLINK_FOLLOW); let symlink_follow = src_flags.contains(types::Lookupflags::SYMLINK_FOLLOW);
if symlink_follow { if symlink_follow {
return Err(Error::invalid_argument() return Err(Error::invalid_argument()
@@ -769,7 +761,7 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
let dir = dir_entry.get_cap(DirCaps::OPEN)?; let dir = dir_entry.get_cap(DirCaps::OPEN)?;
let child_dir = dir.open_dir(symlink_follow, path.deref()).await?; let child_dir = dir.open_dir(symlink_follow, path.deref()).await?;
drop(dir); drop(dir);
let fd = table.push(Box::new(DirEntry::new( let fd = table.push(Arc::new(DirEntry::new(
dir_caps, file_caps, None, child_dir, dir_caps, file_caps, None, child_dir,
)))?; )))?;
Ok(types::Fd::from(fd)) Ok(types::Fd::from(fd))
@@ -789,7 +781,7 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
.open_file(symlink_follow, path.deref(), oflags, read, write, fdflags) .open_file(symlink_follow, path.deref(), oflags, read, write, fdflags)
.await?; .await?;
drop(dir); drop(dir);
let fd = table.push(Box::new(FileEntry::new(file_caps, file)))?; let fd = table.push(Arc::new(FileEntry::new(file_caps, file)))?;
Ok(types::Fd::from(fd)) Ok(types::Fd::from(fd))
} }
} }
@@ -839,12 +831,10 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
dest_path: &GuestPtr<'a, str>, dest_path: &GuestPtr<'a, str>,
) -> Result<(), Error> { ) -> Result<(), Error> {
let table = self.table(); let table = self.table();
let src_dir = table let src_dir = table.get_dir(u32::from(src_fd))?;
.get_dir(u32::from(src_fd))? let src_dir = src_dir.get_cap(DirCaps::RENAME_SOURCE)?;
.get_cap(DirCaps::RENAME_SOURCE)?; let dest_dir = table.get_dir(u32::from(dest_fd))?;
let dest_dir = table let dest_dir = dest_dir.get_cap(DirCaps::RENAME_TARGET)?;
.get_dir(u32::from(dest_fd))?
.get_cap(DirCaps::RENAME_TARGET)?;
src_dir src_dir
.rename( .rename(
src_path.as_cow()?.deref(), src_path.as_cow()?.deref(),
@@ -914,10 +904,11 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
} }
} }
let table = &mut self.table; let table = &self.table;
// We need these refmuts to outlive Poll, which will hold the &mut dyn WasiFile inside // We need these refmuts to outlive Poll, which will hold the &mut dyn WasiFile inside
let mut read_refs: Vec<(&dyn WasiFile, Userdata)> = Vec::new(); let mut read_refs: Vec<(Arc<FileEntry>, Option<Userdata>)> = Vec::new();
let mut write_refs: Vec<(&dyn WasiFile, Userdata)> = Vec::new(); let mut write_refs: Vec<(Arc<FileEntry>, Option<Userdata>)> = Vec::new();
let mut poll = Poll::new(); let mut poll = Poll::new();
let subs = subs.as_array(nsubscriptions); let subs = subs.as_array(nsubscriptions);
@@ -983,25 +974,37 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
}, },
types::SubscriptionU::FdRead(readsub) => { types::SubscriptionU::FdRead(readsub) => {
let fd = readsub.file_descriptor; let fd = readsub.file_descriptor;
let file_ref = table let file_ref = table.get_file(u32::from(fd))?;
.get_file(u32::from(fd))? let _file = file_ref.get_cap(FileCaps::POLL_READWRITE)?;
.get_cap(FileCaps::POLL_READWRITE)?;
read_refs.push((file_ref, sub.userdata.into())); read_refs.push((file_ref, Some(sub.userdata.into())));
} }
types::SubscriptionU::FdWrite(writesub) => { types::SubscriptionU::FdWrite(writesub) => {
let fd = writesub.file_descriptor; let fd = writesub.file_descriptor;
let file_ref = table let file_ref = table.get_file(u32::from(fd))?;
.get_file(u32::from(fd))? let _file = file_ref.get_cap(FileCaps::POLL_READWRITE)?;
.get_cap(FileCaps::POLL_READWRITE)?; write_refs.push((file_ref, Some(sub.userdata.into())));
write_refs.push((file_ref, sub.userdata.into()));
} }
} }
} }
for (f, ud) in read_refs.iter_mut() { let mut read_mut_refs: Vec<(&dyn WasiFile, Userdata)> = Vec::new();
for (file_lock, userdata) in read_refs.iter_mut() {
let file = file_lock.get_cap(FileCaps::POLL_READWRITE)?;
read_mut_refs.push((file, userdata.take().unwrap()));
}
for (f, ud) in read_mut_refs.iter_mut() {
poll.subscribe_read(*f, *ud); poll.subscribe_read(*f, *ud);
} }
for (f, ud) in write_refs.iter_mut() {
let mut write_mut_refs: Vec<(&dyn WasiFile, Userdata)> = Vec::new();
for (file_lock, userdata) in write_refs.iter_mut() {
let file = file_lock.get_cap(FileCaps::POLL_READWRITE)?;
write_mut_refs.push((file, userdata.take().unwrap()));
}
for (f, ud) in write_mut_refs.iter_mut() {
poll.subscribe_write(*f, *ud); poll.subscribe_write(*f, *ud);
} }
@@ -1112,7 +1115,7 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
while copied < buf.len() { while copied < buf.len() {
let len = (buf.len() - copied).min(MAX_SHARED_BUFFER_SIZE as u32); let len = (buf.len() - copied).min(MAX_SHARED_BUFFER_SIZE as u32);
let mut tmp = vec![0; len as usize]; let mut tmp = vec![0; len as usize];
self.random.try_fill_bytes(&mut tmp)?; self.random.lock().unwrap().try_fill_bytes(&mut tmp)?;
let dest = buf let dest = buf
.get_range(copied..copied + len) .get_range(copied..copied + len)
.unwrap() .unwrap()
@@ -1124,7 +1127,7 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
// If the Wasm memory is non-shared, copy directly into the linear // If the Wasm memory is non-shared, copy directly into the linear
// memory. // memory.
let mem = &mut buf.as_slice_mut()?.unwrap(); let mem = &mut buf.as_slice_mut()?.unwrap();
self.random.try_fill_bytes(mem)?; self.random.lock().unwrap().try_fill_bytes(mem)?;
} }
Ok(()) Ok(())
} }
@@ -1135,9 +1138,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
flags: types::Fdflags, flags: types::Fdflags,
) -> Result<types::Fd, Error> { ) -> Result<types::Fd, Error> {
let table = self.table(); let table = self.table();
let f = table let f = table.get_file(u32::from(fd))?;
.get_file_mut(u32::from(fd))? let f = f.get_cap(FileCaps::READ)?;
.get_cap_mut(FileCaps::READ)?;
let file = f.sock_accept(FdFlags::from(flags)).await?; let file = f.sock_accept(FdFlags::from(flags)).await?;
let file_caps = FileCaps::READ let file_caps = FileCaps::READ
@@ -1146,7 +1148,7 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
| FileCaps::POLL_READWRITE | FileCaps::POLL_READWRITE
| FileCaps::FILESTAT_GET; | FileCaps::FILESTAT_GET;
let fd = table.push(Box::new(FileEntry::new(file_caps, file)))?; let fd = table.push(Arc::new(FileEntry::new(file_caps, file)))?;
Ok(types::Fd::from(fd)) Ok(types::Fd::from(fd))
} }
@@ -1156,10 +1158,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
ri_data: &types::IovecArray<'a>, ri_data: &types::IovecArray<'a>,
ri_flags: types::Riflags, ri_flags: types::Riflags,
) -> Result<(types::Size, types::Roflags), Error> { ) -> Result<(types::Size, types::Roflags), Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::READ)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::READ)?;
let iovs: Vec<wiggle::GuestPtr<[u8]>> = ri_data let iovs: Vec<wiggle::GuestPtr<[u8]>> = ri_data
.iter() .iter()
@@ -1231,10 +1231,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
si_data: &types::CiovecArray<'a>, si_data: &types::CiovecArray<'a>,
_si_flags: types::Siflags, _si_flags: types::Siflags,
) -> Result<types::Size, Error> { ) -> Result<types::Size, Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::WRITE)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::WRITE)?;
let guest_slices: Vec<wiggle::GuestCow<u8>> = si_data let guest_slices: Vec<wiggle::GuestCow<u8>> = si_data
.iter() .iter()
@@ -1255,10 +1253,8 @@ impl wasi_snapshot_preview1::WasiSnapshotPreview1 for WasiCtx {
} }
async fn sock_shutdown(&mut self, fd: types::Fd, how: types::Sdflags) -> Result<(), Error> { async fn sock_shutdown(&mut self, fd: types::Fd, how: types::Sdflags) -> Result<(), Error> {
let f = self let f = self.table().get_file(u32::from(fd))?;
.table() let f = f.get_cap(FileCaps::FDSTAT_SET_FLAGS)?;
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::FDSTAT_SET_FLAGS)?;
f.sock_shutdown(SdFlags::from(how)).await f.sock_shutdown(SdFlags::from(how)).await
} }

86
crates/wasi-common/src/table.rs
View File

@@ -1,6 +1,7 @@
use crate::{Error, ErrorExt}; use crate::{Error, ErrorExt};
use std::any::Any; use std::any::Any;
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::{Arc, RwLock};
/// The `Table` type is designed to map u32 handles to resources. The table is now part of the /// The `Table` type is designed to map u32 handles to resources. The table is now part of the
/// public interface to a `WasiCtx` - it is reference counted so that it can be shared beyond a /// public interface to a `WasiCtx` - it is reference counted so that it can be shared beyond a
@@ -9,84 +10,105 @@ use std::collections::HashMap;
/// ///
/// The `Table` type is intended to model how the Interface Types concept of Resources is shaping /// The `Table` type is intended to model how the Interface Types concept of Resources is shaping
/// up. Right now it is just an approximation. /// up. Right now it is just an approximation.
pub struct Table { pub struct Table(RwLock<Inner>);
map: HashMap<u32, Box<dyn Any + Send + Sync>>,
struct Inner {
map: HashMap<u32, Arc<dyn Any + Send + Sync>>,
next_key: u32, next_key: u32,
} }
impl Table { impl Table {
/// Create an empty table. New insertions will begin at 3, above stdio. /// Create an empty table. New insertions will begin at 3, above stdio.
pub fn new() -> Self { pub fn new() -> Self {
Table { Table(RwLock::new(Inner {
map: HashMap::new(), map: HashMap::new(),
next_key: 3, // 0, 1 and 2 are reserved for stdio next_key: 3, // 0, 1 and 2 are reserved for stdio
} }))
} }
/// Insert a resource at a certain index. /// Insert a resource at a certain index.
pub fn insert_at(&mut self, key: u32, a: Box<dyn Any + Send + Sync>) { pub fn insert_at<T: Any + Send + Sync>(&self, key: u32, a: Arc<T>) {
self.map.insert(key, a); self.0.write().unwrap().map.insert(key, a);
} }
/// Insert a resource at the next available index. /// Insert a resource at the next available index.
pub fn push(&mut self, a: Box<dyn Any + Send + Sync>) -> Result<u32, Error> { pub fn push<T: Any + Send + Sync>(&self, a: Arc<T>) -> Result<u32, Error> {
let mut inner = self.0.write().unwrap();
// NOTE: The performance of this new key calculation could be very bad once keys wrap // NOTE: The performance of this new key calculation could be very bad once keys wrap
// around. // around.
if self.map.len() == u32::MAX as usize { if inner.map.len() == u32::MAX as usize {
return Err(Error::trap(anyhow::Error::msg("table has no free keys"))); return Err(Error::trap(anyhow::Error::msg("table has no free keys")));
} }
loop { loop {
let key = self.next_key; let key = inner.next_key;
self.next_key = self.next_key.wrapping_add(1); inner.next_key += 1;
if self.map.contains_key(&key) { if inner.map.contains_key(&key) {
continue; continue;
} }
self.map.insert(key, a); inner.map.insert(key, a);
return Ok(key); return Ok(key);
} }
} }
/// Check if the table has a resource at the given index. /// Check if the table has a resource at the given index.
pub fn contains_key(&self, key: u32) -> bool { pub fn contains_key(&self, key: u32) -> bool {
self.map.contains_key(&key) self.0.read().unwrap().map.contains_key(&key)
} }
/// Check if the resource at a given index can be downcast to a given type. /// Check if the resource at a given index can be downcast to a given type.
/// Note: this will always fail if the resource is already borrowed. /// Note: this will always fail if the resource is already borrowed.
pub fn is<T: Any + Sized>(&self, key: u32) -> bool { pub fn is<T: Any + Sized>(&self, key: u32) -> bool {
if let Some(r) = self.map.get(&key) { if let Some(r) = self.0.read().unwrap().map.get(&key) {
r.is::<T>() r.is::<T>()
} else { } else {
false false
} }
} }
/// Get an immutable reference to a resource of a given type at a given index. Multiple /// Get an Arc reference to a resource of a given type at a given index. Multiple
/// immutable references can be borrowed at any given time. Borrow failure /// immutable references can be borrowed at any given time.
/// results in a trapping error. pub fn get<T: Any + Send + Sync + Sized>(&self, key: u32) -> Result<Arc<T>, Error> {
pub fn get<T: Any + Sized>(&self, key: u32) -> Result<&T, Error> { if let Some(r) = self.0.read().unwrap().map.get(&key).cloned() {
if let Some(r) = self.map.get(&key) { r.downcast::<T>()
r.downcast_ref::<T>() .map_err(|_| Error::badf().context("element is a different type"))
.ok_or_else(|| Error::badf().context("element is a different type"))
} else { } else {
Err(Error::badf().context("key not in table")) Err(Error::badf().context("key not in table"))
} }
} }
/// Get a mutable reference to a resource of a given type at a given index. Only one mutable /// Get a mutable reference to a resource of a given type at a given index.
/// reference can be borrowed at any given time. Borrow failure results in a trapping error. /// Only one such reference can be borrowed at any given time.
pub fn get_mut<T: Any + Sized>(&mut self, key: u32) -> Result<&mut T, Error> { pub fn get_mut<T: Any>(&mut self, key: u32) -> Result<&mut T, Error> {
if let Some(r) = self.map.get_mut(&key) { let entry = match self.0.get_mut().unwrap().map.get_mut(&key) {
r.downcast_mut::<T>() Some(entry) => entry,
.ok_or_else(|| Error::badf().context("element is a different type")) None => return Err(Error::badf().context("key not in table")),
} else { };
Err(Error::badf().context("key not in table")) let entry = match Arc::get_mut(entry) {
} Some(entry) => entry,
None => return Err(Error::badf().context("cannot mutably borrow shared file")),
};
entry
.downcast_mut::<T>()
.ok_or_else(|| Error::badf().context("element is a different type"))
} }
/// Remove a resource at a given index from the table. Returns the resource /// Remove a resource at a given index from the table. Returns the resource
/// if it was present. /// if it was present.
pub fn delete(&mut self, key: u32) -> Option<Box<dyn Any + Send + Sync>> { pub fn delete<T: Any + Send + Sync>(&self, key: u32) -> Option<Arc<T>> {
self.map.remove(&key) self.0
.write()
.unwrap()
.map
.remove(&key)
.map(|r| r.downcast::<T>().unwrap())
}
/// Remove a resource at a given index from the table. Returns the resource
/// if it was present.
pub fn renumber(&self, from: u32, to: u32) -> Result<(), Error> {
let map = &mut self.0.write().unwrap().map;
let from_entry = map.remove(&from).ok_or(Error::badf())?;
map.insert(to, from_entry);
Ok(())
} }
} }

46
crates/wasi-common/tokio/src/file.rs
View File

@@ -4,6 +4,7 @@ use io_extras::os::windows::{AsRawHandleOrSocket, RawHandleOrSocket};
#[cfg(not(windows))] #[cfg(not(windows))]
use io_lifetimes::AsFd; use io_lifetimes::AsFd;
use std::any::Any; use std::any::Any;
use std::borrow::Borrow;
use std::io; use std::io;
use wasi_common::{ use wasi_common::{
file::{Advice, FdFlags, FileType, Filestat, WasiFile}, file::{Advice, FdFlags, FileType, Filestat, WasiFile},
@@ -98,78 +99,77 @@ macro_rules! wasi_file_impl {
fn pollable(&self) -> Option<rustix::fd::BorrowedFd> { fn pollable(&self) -> Option<rustix::fd::BorrowedFd> {
Some(self.0.as_fd()) Some(self.0.as_fd())
} }
#[cfg(windows)] #[cfg(windows)]
fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> { fn pollable(&self) -> Option<io_extras::os::windows::RawHandleOrSocket> {
Some(self.0.as_raw_handle_or_socket()) Some(self.0.as_raw_handle_or_socket())
} }
async fn datasync(&mut self) -> Result<(), Error> { async fn datasync(&self) -> Result<(), Error> {
block_on_dummy_executor(|| self.0.datasync()) block_on_dummy_executor(|| self.0.datasync())
} }
async fn sync(&mut self) -> Result<(), Error> { async fn sync(&self) -> Result<(), Error> {
block_on_dummy_executor(|| self.0.sync()) block_on_dummy_executor(|| self.0.sync())
} }
async fn get_filetype(&mut self) -> Result<FileType, Error> { async fn get_filetype(&self) -> Result<FileType, Error> {
block_on_dummy_executor(|| self.0.get_filetype()) block_on_dummy_executor(|| self.0.get_filetype())
} }
async fn get_fdflags(&mut self) -> Result<FdFlags, Error> { async fn get_fdflags(&self) -> Result<FdFlags, Error> {
block_on_dummy_executor(|| self.0.get_fdflags()) block_on_dummy_executor(|| self.0.get_fdflags())
} }
async fn set_fdflags(&mut self, fdflags: FdFlags) -> Result<(), Error> { async fn set_fdflags(&mut self, fdflags: FdFlags) -> Result<(), Error> {
block_on_dummy_executor(|| self.0.set_fdflags(fdflags)) block_on_dummy_executor(|| self.0.set_fdflags(fdflags))
} }
async fn get_filestat(&mut self) -> Result<Filestat, Error> { async fn get_filestat(&self) -> Result<Filestat, Error> {
block_on_dummy_executor(|| self.0.get_filestat()) block_on_dummy_executor(|| self.0.get_filestat())
} }
async fn set_filestat_size(&mut self, size: u64) -> Result<(), Error> { async fn set_filestat_size(&self, size: u64) -> Result<(), Error> {
block_on_dummy_executor(move || self.0.set_filestat_size(size)) block_on_dummy_executor(move || self.0.set_filestat_size(size))
} }
async fn advise(&mut self, offset: u64, len: u64, advice: Advice) -> Result<(), Error> { async fn advise(&self, offset: u64, len: u64, advice: Advice) -> Result<(), Error> {
block_on_dummy_executor(move || self.0.advise(offset, len, advice)) block_on_dummy_executor(move || self.0.advise(offset, len, advice))
} }
async fn allocate(&mut self, offset: u64, len: u64) -> Result<(), Error> { async fn allocate(&self, offset: u64, len: u64) -> Result<(), Error> {
block_on_dummy_executor(move || self.0.allocate(offset, len)) block_on_dummy_executor(move || self.0.allocate(offset, len))
} }
async fn read_vectored<'a>( async fn read_vectored<'a>(
&mut self, &self,
bufs: &mut [io::IoSliceMut<'a>], bufs: &mut [io::IoSliceMut<'a>],
) -> Result<u64, Error> { ) -> Result<u64, Error> {
block_on_dummy_executor(move || self.0.read_vectored(bufs)) block_on_dummy_executor(move || self.0.read_vectored(bufs))
} }
async fn read_vectored_at<'a>( async fn read_vectored_at<'a>(
&mut self, &self,
bufs: &mut [io::IoSliceMut<'a>], bufs: &mut [io::IoSliceMut<'a>],
offset: u64, offset: u64,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
block_on_dummy_executor(move || self.0.read_vectored_at(bufs, offset)) block_on_dummy_executor(move || self.0.read_vectored_at(bufs, offset))
} }
async fn write_vectored<'a>(&mut self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> { async fn write_vectored<'a>(&self, bufs: &[io::IoSlice<'a>]) -> Result<u64, Error> {
block_on_dummy_executor(move || self.0.write_vectored(bufs)) block_on_dummy_executor(move || self.0.write_vectored(bufs))
} }
async fn write_vectored_at<'a>( async fn write_vectored_at<'a>(
&mut self, &self,
bufs: &[io::IoSlice<'a>], bufs: &[io::IoSlice<'a>],
offset: u64, offset: u64,
) -> Result<u64, Error> { ) -> Result<u64, Error> {
block_on_dummy_executor(move || self.0.write_vectored_at(bufs, offset)) block_on_dummy_executor(move || self.0.write_vectored_at(bufs, offset))
} }
async fn seek(&mut self, pos: std::io::SeekFrom) -> Result<u64, Error> { async fn seek(&self, pos: std::io::SeekFrom) -> Result<u64, Error> {
block_on_dummy_executor(move || self.0.seek(pos)) block_on_dummy_executor(move || self.0.seek(pos))
} }
async fn peek(&mut self, buf: &mut [u8]) -> Result<u64, Error> { async fn peek(&self, buf: &mut [u8]) -> Result<u64, Error> {
block_on_dummy_executor(move || self.0.peek(buf)) block_on_dummy_executor(move || self.0.peek(buf))
} }
async fn set_times( async fn set_times(
&mut self, &self,
atime: Option<wasi_common::SystemTimeSpec>, atime: Option<wasi_common::SystemTimeSpec>,
mtime: Option<wasi_common::SystemTimeSpec>, mtime: Option<wasi_common::SystemTimeSpec>,
) -> Result<(), Error> { ) -> Result<(), Error> {
block_on_dummy_executor(move || self.0.set_times(atime, mtime)) block_on_dummy_executor(move || self.0.set_times(atime, mtime))
} }
async fn num_ready_bytes(&self) -> Result<u64, Error> { fn num_ready_bytes(&self) -> Result<u64, Error> {
block_on_dummy_executor(|| self.0.num_ready_bytes()) self.0.num_ready_bytes()
} }
fn isatty(&mut self) -> bool { fn isatty(&self) -> bool {
self.0.isatty() self.0.isatty()
} }
@@ -182,7 +182,7 @@ macro_rules! wasi_file_impl {
// lifetime of the AsyncFd. // lifetime of the AsyncFd.
use std::os::unix::io::AsRawFd; use std::os::unix::io::AsRawFd;
use tokio::io::{unix::AsyncFd, Interest}; use tokio::io::{unix::AsyncFd, Interest};
let rawfd = self.0.as_fd().as_raw_fd(); let rawfd = self.0.borrow().as_fd().as_raw_fd();
match AsyncFd::with_interest(rawfd, Interest::READABLE) { match AsyncFd::with_interest(rawfd, Interest::READABLE) {
Ok(asyncfd) => { Ok(asyncfd) => {
let _ = asyncfd.readable().await?; let _ = asyncfd.readable().await?;
@@ -206,7 +206,7 @@ macro_rules! wasi_file_impl {
// lifetime of the AsyncFd. // lifetime of the AsyncFd.
use std::os::unix::io::AsRawFd; use std::os::unix::io::AsRawFd;
use tokio::io::{unix::AsyncFd, Interest}; use tokio::io::{unix::AsyncFd, Interest};
let rawfd = self.0.as_fd().as_raw_fd(); let rawfd = self.0.borrow().as_fd().as_raw_fd();
match AsyncFd::with_interest(rawfd, Interest::WRITABLE) { match AsyncFd::with_interest(rawfd, Interest::WRITABLE) {
Ok(asyncfd) => { Ok(asyncfd) => {
let _ = asyncfd.writable().await?; let _ = asyncfd.writable().await?;
@@ -221,7 +221,7 @@ macro_rules! wasi_file_impl {
} }
} }
async fn sock_accept(&mut self, fdflags: FdFlags) -> Result<Box<dyn WasiFile>, Error> { async fn sock_accept(&self, fdflags: FdFlags) -> Result<Box<dyn WasiFile>, Error> {
block_on_dummy_executor(|| self.0.sock_accept(fdflags)) block_on_dummy_executor(|| self.0.sock_accept(fdflags))
} }
} }
@@ -229,7 +229,7 @@ macro_rules! wasi_file_impl {
impl AsRawHandleOrSocket for $ty { impl AsRawHandleOrSocket for $ty {
#[inline] #[inline]
fn as_raw_handle_or_socket(&self) -> RawHandleOrSocket { fn as_raw_handle_or_socket(&self) -> RawHandleOrSocket {
self.0.as_raw_handle_or_socket() self.0.borrow().as_raw_handle_or_socket()
} }
} }
}; };

10
crates/wasi-common/tokio/src/lib.rs
View File

@@ -62,15 +62,15 @@ impl WasiCtxBuilder {
} }
Ok(self) Ok(self)
} }
pub fn stdin(mut self, f: Box<dyn WasiFile>) -> Self { pub fn stdin(self, f: Box<dyn WasiFile>) -> Self {
self.0.set_stdin(f); self.0.set_stdin(f);
self self
} }
pub fn stdout(mut self, f: Box<dyn WasiFile>) -> Self { pub fn stdout(self, f: Box<dyn WasiFile>) -> Self {
self.0.set_stdout(f); self.0.set_stdout(f);
self self
} }
pub fn stderr(mut self, f: Box<dyn WasiFile>) -> Self { pub fn stderr(self, f: Box<dyn WasiFile>) -> Self {
self.0.set_stderr(f); self.0.set_stderr(f);
self self
} }
@@ -87,7 +87,7 @@ impl WasiCtxBuilder {
self.inherit_stdin().inherit_stdout().inherit_stderr() self.inherit_stdin().inherit_stdout().inherit_stderr()
} }
pub fn preopened_dir( pub fn preopened_dir(
mut self, self,
dir: cap_std::fs::Dir, dir: cap_std::fs::Dir,
guest_path: impl AsRef<Path>, guest_path: impl AsRef<Path>,
) -> Result<Self, Error> { ) -> Result<Self, Error> {
@@ -95,7 +95,7 @@ impl WasiCtxBuilder {
self.0.push_preopened_dir(dir, guest_path)?; self.0.push_preopened_dir(dir, guest_path)?;
Ok(self) Ok(self)
} }
pub fn preopened_socket(mut self, fd: u32, socket: impl Into<Socket>) -> Result<Self, Error> { pub fn preopened_socket(self, fd: u32, socket: impl Into<Socket>) -> Result<Self, Error> {
let socket: Socket = socket.into(); let socket: Socket = socket.into();
let file: Box<dyn WasiFile> = socket.into(); let file: Box<dyn WasiFile> = socket.into();

1
crates/wasi-common/tokio/src/sched/unix.rs
View File

@@ -63,7 +63,6 @@ pub async fn poll_oneoff<'a>(poll: &mut Poll<'a>) -> Result<(), Error> {
f.complete( f.complete(
f.file f.file
.num_ready_bytes() .num_ready_bytes()
.await
.map_err(|e| e.context("read num_ready_bytes"))?, .map_err(|e| e.context("read num_ready_bytes"))?,
RwEventFlags::empty(), RwEventFlags::empty(),
); );

2
crates/wasi-common/tokio/tests/poll_oneoff.rs
View File

@@ -20,7 +20,7 @@ async fn empty_file_readable() -> Result<(), Error> {
let d = workspace.open_dir("d").context("open dir")?; let d = workspace.open_dir("d").context("open dir")?;
let d = Dir::from_cap_std(d); let d = Dir::from_cap_std(d);
let mut f = d let f = d
.open_file(false, "f", OFlags::CREATE, false, true, FdFlags::empty()) .open_file(false, "f", OFlags::CREATE, false, true, FdFlags::empty())
.await .await
.context("create writable file f")?; .context("create writable file f")?;

4
crates/wasi-nn/src/api.rs
View File

@@ -7,7 +7,7 @@ use thiserror::Error;
use wiggle::GuestError; use wiggle::GuestError;
/// A [Backend] contains the necessary state to load [BackendGraph]s. /// A [Backend] contains the necessary state to load [BackendGraph]s.
pub(crate) trait Backend: Send { pub(crate) trait Backend: Send + Sync {
fn name(&self) -> &str; fn name(&self) -> &str;
fn load( fn load(
&mut self, &mut self,
@@ -18,7 +18,7 @@ pub(crate) trait Backend: Send {
/// A [BackendGraph] can create [BackendExecutionContext]s; this is the backing /// A [BackendGraph] can create [BackendExecutionContext]s; this is the backing
/// implementation for a [crate::witx::types::Graph]. /// implementation for a [crate::witx::types::Graph].
pub(crate) trait BackendGraph: Send { pub(crate) trait BackendGraph: Send + Sync {
fn init_execution_context(&mut self) -> Result<Box<dyn BackendExecutionContext>, BackendError>; fn init_execution_context(&mut self) -> Result<Box<dyn BackendExecutionContext>, BackendError>;
} }

7
crates/wasi-nn/src/openvino.rs
View File

@@ -1,4 +1,5 @@
//! Implements the wasi-nn API. //! Implements the wasi-nn API.
use crate::api::{Backend, BackendError, BackendExecutionContext, BackendGraph}; use crate::api::{Backend, BackendError, BackendExecutionContext, BackendGraph};
use crate::witx::types::{ExecutionTarget, GraphBuilderArray, Tensor, TensorType}; use crate::witx::types::{ExecutionTarget, GraphBuilderArray, Tensor, TensorType};
use openvino::{InferenceError, Layout, Precision, SetupError, TensorDesc}; use openvino::{InferenceError, Layout, Precision, SetupError, TensorDesc};
@@ -7,6 +8,9 @@ use std::sync::Arc;
#[derive(Default)] #[derive(Default)]
pub(crate) struct OpenvinoBackend(Option<openvino::Core>); pub(crate) struct OpenvinoBackend(Option<openvino::Core>);
unsafe impl Send for OpenvinoBackend {}
unsafe impl Sync for OpenvinoBackend {}
impl Backend for OpenvinoBackend { impl Backend for OpenvinoBackend {
fn name(&self) -> &str { fn name(&self) -> &str {
"openvino" "openvino"
@@ -65,6 +69,9 @@ impl Backend for OpenvinoBackend {
struct OpenvinoGraph(Arc<openvino::CNNNetwork>, openvino::ExecutableNetwork); struct OpenvinoGraph(Arc<openvino::CNNNetwork>, openvino::ExecutableNetwork);
unsafe impl Send for OpenvinoGraph {}
unsafe impl Sync for OpenvinoGraph {}
impl BackendGraph for OpenvinoGraph { impl BackendGraph for OpenvinoGraph {
fn init_execution_context(&mut self) -> Result<Box<dyn BackendExecutionContext>, BackendError> { fn init_execution_context(&mut self) -> Result<Box<dyn BackendExecutionContext>, BackendError> {
let infer_request = self.1.create_infer_request()?; let infer_request = self.1.create_infer_request()?;

23
crates/wasi-threads/Cargo.toml Normal file
View File

@@ -0,0 +1,23 @@
[package]
name = "wasmtime-wasi-threads"
version.workspace = true
authors.workspace = true
description = "Wasmtime implementation of the wasi-threads API"
documentation = "https://docs.rs/wasmtime-wasi-nn"
license = "Apache-2.0 WITH LLVM-exception"
categories = ["wasm", "parallelism", "threads"]
keywords = ["webassembly", "wasm", "neural-network"]
repository = "https://github.com/bytecodealliance/wasmtime"
readme = "README.md"
edition.workspace = true
[dependencies]
anyhow = { workspace = true }
log = { workspace = true }
rand = "0.8"
wasi-common = { workspace = true }
wasmtime = { workspace = true }
wasmtime-wasi = { workspace = true, features = ["exit"] }
[badges]
maintenance = { status = "experimental" }

12
crates/wasi-threads/README.md Normal file
View File

@@ -0,0 +1,12 @@
# wasmtime-wasi-threads
Implement the `wasi-threads` [specification] in Wasmtime.
[specification]: https://github.com/WebAssembly/wasi-threads
> Note: this crate is experimental and not yet suitable for use in multi-tenant
> embeddings. As specified, a trap or WASI exit in one thread must end execution
> for all threads. Due to the complexity of stopping threads, however, this
> implementation currently exits the process entirely. This will work for some
> use cases (e.g., CLI usage) but not for embedders. This warning can be removed
> once a suitable mechanism is implemented that avoids exiting the process.

159
crates/wasi-threads/src/lib.rs Normal file
View File

@@ -0,0 +1,159 @@
//! Implement [`wasi-threads`].
//!
//! [`wasi-threads`]: https://github.com/WebAssembly/wasi-threads
use anyhow::{anyhow, bail, Result};
use rand::Rng;
use std::panic::{catch_unwind, AssertUnwindSafe};
use std::sync::Arc;
use std::thread;
use wasmtime::{Caller, Linker, Module, SharedMemory, Store, ValType};
use wasmtime_wasi::maybe_exit_on_error;
// This name is a function export designated by the wasi-threads specification:
// https://github.com/WebAssembly/wasi-threads/#detailed-design-discussion
const WASI_ENTRY_POINT: &str = "wasi_thread_start";
pub struct WasiThreadsCtx<T> {
module: Module,
linker: Arc<Linker<T>>,
}
impl<T: Clone + Send + 'static> WasiThreadsCtx<T> {
pub fn new(module: Module, linker: Arc<Linker<T>>) -> Result<Self> {
if !has_wasi_entry_point(&module) {
bail!(
"failed to find wasi-threads entry point function: {}",
WASI_ENTRY_POINT
);
}
Ok(Self { module, linker })
}
pub fn spawn(&self, host: T, thread_start_arg: i32) -> Result<i32> {
let module = self.module.clone();
let linker = self.linker.clone();
// Start a Rust thread running a new instance of the current module.
let wasi_thread_id = random_thread_id();
let builder = thread::Builder::new().name(format!("wasi-thread-{}", wasi_thread_id));
builder.spawn(move || {
// Catch any panic failures in host code; e.g., if a WASI module
// were to crash, we want all threads to exit, not just this one.
let result = catch_unwind(AssertUnwindSafe(|| {
// Each new instance is created in its own store.
let mut store = Store::new(&module.engine(), host);
// Ideally, we would have already checked much earlier (e.g.,
// `new`) whether the module can be instantiated. Because
// `Linker::instantiate_pre` requires a `Store` and that is only
// available now. TODO:
// https://github.com/bytecodealliance/wasmtime/issues/5675.
let instance = linker.instantiate(&mut store, &module).expect(&format!(
"wasi-thread-{} exited unsuccessfully: failed to instantiate",
wasi_thread_id
));
let thread_entry_point = instance
.get_typed_func::<(i32, i32), ()>(&mut store, WASI_ENTRY_POINT)
.unwrap();
// Start the thread's entry point. Any traps or calls to
// `proc_exit`, by specification, should end execution for all
// threads. This code uses `process::exit` to do so, which is what
// the user expects from the CLI but probably not in a Wasmtime
// embedding.
log::trace!(
"spawned thread id = {}; calling start function `{}` with: {}",
wasi_thread_id,
WASI_ENTRY_POINT,
thread_start_arg
);
match thread_entry_point.call(&mut store, (wasi_thread_id, thread_start_arg)) {
Ok(_) => log::trace!("exiting thread id = {} normally", wasi_thread_id),
Err(e) => {
log::trace!("exiting thread id = {} due to error", wasi_thread_id);
let e = maybe_exit_on_error(e);
eprintln!("Error: {:?}", e);
std::process::exit(1);
}
}
}));
if let Err(e) = result {
eprintln!("wasi-thread-{} panicked: {:?}", wasi_thread_id, e);
std::process::exit(1);
}
})?;
Ok(wasi_thread_id)
}
}
/// Helper for generating valid WASI thread IDs (TID).
///
/// Callers of `wasi_thread_spawn` expect a TID >=0 to indicate a successful
/// spawning of the thread whereas a negative return value indicates an
/// failure to spawn.
fn random_thread_id() -> i32 {
let tid: u32 = rand::thread_rng().gen();
(tid >> 1) as i32
}
/// Manually add the WASI `thread_spawn` function to the linker.
///
/// It is unclear what namespace the `wasi-threads` proposal should live under:
/// it is not clear if it should be included in any of the `preview*` releases
/// so for the time being its module namespace is simply `"wasi"` (TODO).
pub fn add_to_linker<T: Clone + Send + 'static>(
linker: &mut wasmtime::Linker<T>,
store: &wasmtime::Store<T>,
module: &Module,
get_cx: impl Fn(&mut T) -> &WasiThreadsCtx<T> + Send + Sync + Copy + 'static,
) -> anyhow::Result<SharedMemory> {
linker.func_wrap(
"wasi",
"thread_spawn",
move |mut caller: Caller<'_, T>, start_arg: i32| -> i32 {
log::trace!("new thread requested via `wasi::thread_spawn` call");
let host = caller.data().clone();
let ctx = get_cx(caller.data_mut());
match ctx.spawn(host, start_arg) {
Ok(thread_id) => {
assert!(thread_id >= 0, "thread_id = {}", thread_id);
thread_id
}
Err(e) => {
log::error!("failed to spawn thread: {}", e);
-1
}
}
},
)?;
// Find the shared memory import and satisfy it with a newly-created shared
// memory import. This currently does not handle multiple memories (TODO).
for import in module.imports() {
if let Some(m) = import.ty().memory() {
if m.is_shared() {
let mem = SharedMemory::new(module.engine(), m.clone())?;
linker.define(store, import.module(), import.name(), mem.clone())?;
return Ok(mem);
}
}
}
Err(anyhow!(
"unable to link a shared memory import to the module; a `wasi-threads` \
module should import a single shared memory as \"memory\""
))
}
fn has_wasi_entry_point(module: &Module) -> bool {
module
.get_export(WASI_ENTRY_POINT)
.and_then(|t| t.func().cloned())
.and_then(|t| {
let params: Vec<ValType> = t.params().collect();
Some(params == [ValType::I32, ValType::I32] && t.results().len() == 0)
})
.unwrap_or(false)
}

2
crates/wasi/Cargo.toml
View File

@@ -13,6 +13,7 @@ include = ["src/**/*", "README.md", "LICENSE", "build.rs"]
build = "build.rs" build = "build.rs"
[dependencies] [dependencies]
libc = "0.2.60"
wasi-common = { workspace = true } wasi-common = { workspace = true }
wasi-cap-std-sync = { workspace = true, optional = true } wasi-cap-std-sync = { workspace = true, optional = true }
wasi-tokio = { workspace = true, optional = true } wasi-tokio = { workspace = true, optional = true }
@@ -24,3 +25,4 @@ anyhow = { workspace = true }
default = ["sync"] default = ["sync"]
sync = ["wasi-cap-std-sync"] sync = ["wasi-cap-std-sync"]
tokio = ["wasi-tokio", "wasmtime/async", "wiggle/wasmtime_async"] tokio = ["wasi-tokio", "wasmtime/async", "wiggle/wasmtime_async"]
exit = []

44
crates/wasi/src/lib.rs
View File

@@ -82,3 +82,47 @@ pub mod snapshots {
} }
} }
} }
/// Exit the process with a conventional OS error code as long as Wasmtime
/// understands the error. If the error is not an `I32Exit` or `Trap`, return
/// the error back to the caller for it to decide what to do.
///
/// Note: this function is designed for usage where it is acceptable for
/// Wasmtime failures to terminate the parent process, such as in the Wasmtime
/// CLI; this would not be suitable for use in multi-tenant embeddings.
#[cfg(feature = "exit")]
pub fn maybe_exit_on_error(e: anyhow::Error) -> anyhow::Error {
use std::process;
use wasmtime::Trap;
// If a specific WASI error code was requested then that's
// forwarded through to the process here without printing any
// extra error information.
if let Some(exit) = e.downcast_ref::<I32Exit>() {
// Print the error message in the usual way.
// On Windows, exit status 3 indicates an abort (see below),
// so return 1 indicating a non-zero status to avoid ambiguity.
if cfg!(windows) && exit.0 >= 3 {
process::exit(1);
}
process::exit(exit.0);
}
// If the program exited because of a trap, return an error code
// to the outside environment indicating a more severe problem
// than a simple failure.
if e.is::<Trap>() {
eprintln!("Error: {:?}", e);
if cfg!(unix) {
// On Unix, return the error code of an abort.
process::exit(128 + libc::SIGABRT);
} else if cfg!(windows) {
// On Windows, return 3.
// https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/abort?view=vs-2019
process::exit(3);
}
}
e
}

6
scripts/publish.rs
View File

@@ -64,8 +64,9 @@ const CRATES_TO_PUBLISH: &[&str] = &[
"wasi-tokio", "wasi-tokio",
// other misc wasmtime crates // other misc wasmtime crates
"wasmtime-wasi", "wasmtime-wasi",
"wasmtime-wasi-nn",
"wasmtime-wasi-crypto", "wasmtime-wasi-crypto",
"wasmtime-wasi-nn",
"wasmtime-wasi-threads",
"wasmtime-wast", "wasmtime-wast",
"wasmtime-cli-flags", "wasmtime-cli-flags",
"wasmtime-cli", "wasmtime-cli",
@@ -84,8 +85,9 @@ const PUBLIC_CRATES: &[&str] = &[
// patch releases. // patch releases.
"wasmtime", "wasmtime",
"wasmtime-wasi", "wasmtime-wasi",
"wasmtime-wasi-nn",
"wasmtime-wasi-crypto", "wasmtime-wasi-crypto",
"wasmtime-wasi-nn",
"wasmtime-wasi-threads",
"wasmtime-cli", "wasmtime-cli",
// all cranelift crates are considered "public" in that they can't // all cranelift crates are considered "public" in that they can't
// have breaking API changes in patch releases // have breaking API changes in patch releases

159
src/commands/run.rs
View File

@@ -3,17 +3,17 @@
use anyhow::{anyhow, bail, Context as _, Result}; use anyhow::{anyhow, bail, Context as _, Result};
use clap::Parser; use clap::Parser;
use once_cell::sync::Lazy; use once_cell::sync::Lazy;
use std::ffi::OsStr;
use std::path::{Component, Path, PathBuf};
use std::thread; use std::thread;
use std::time::Duration; use std::time::Duration;
use std::{ use wasmtime::{Engine, Func, Linker, Module, Store, Val, ValType};
ffi::OsStr,
path::{Component, Path, PathBuf},
process,
};
use wasmtime::{Engine, Func, Linker, Module, Store, Trap, Val, ValType};
use wasmtime_cli_flags::{CommonOptions, WasiModules}; use wasmtime_cli_flags::{CommonOptions, WasiModules};
use wasmtime_wasi::maybe_exit_on_error;
use wasmtime_wasi::sync::{ambient_authority, Dir, TcpListener, WasiCtxBuilder}; use wasmtime_wasi::sync::{ambient_authority, Dir, TcpListener, WasiCtxBuilder};
use wasmtime_wasi::I32Exit;
#[cfg(any(feature = "wasi-crypto", feature = "wasi-nn", feature = "wasi-threads"))]
use std::sync::Arc;
#[cfg(feature = "wasi-nn")] #[cfg(feature = "wasi-nn")]
use wasmtime_wasi_nn::WasiNnCtx; use wasmtime_wasi_nn::WasiNnCtx;
@@ -21,6 +21,9 @@ use wasmtime_wasi_nn::WasiNnCtx;
#[cfg(feature = "wasi-crypto")] #[cfg(feature = "wasi-crypto")]
use wasmtime_wasi_crypto::WasiCryptoCtx; use wasmtime_wasi_crypto::WasiCryptoCtx;
#[cfg(feature = "wasi-threads")]
use wasmtime_wasi_threads::WasiThreadsCtx;
fn parse_module(s: &OsStr) -> anyhow::Result<PathBuf> { fn parse_module(s: &OsStr) -> anyhow::Result<PathBuf> {
// Do not accept wasmtime subcommand names as the module name // Do not accept wasmtime subcommand names as the module name
match s.to_str() { match s.to_str() {
@@ -164,13 +167,6 @@ impl RunCommand {
config.epoch_interruption(true); config.epoch_interruption(true);
} }
let engine = Engine::new(&config)?; let engine = Engine::new(&config)?;
let mut store = Store::new(&engine, Host::default());
// If fuel has been configured, we want to add the configured
// fuel amount to this store.
if let Some(fuel) = self.common.fuel {
store.add_fuel(fuel)?;
}
let preopen_sockets = self.compute_preopen_sockets()?; let preopen_sockets = self.compute_preopen_sockets()?;
@@ -181,9 +177,15 @@ impl RunCommand {
let mut linker = Linker::new(&engine); let mut linker = Linker::new(&engine);
linker.allow_unknown_exports(self.allow_unknown_exports); linker.allow_unknown_exports(self.allow_unknown_exports);
// Read the wasm module binary either as `*.wat` or a raw binary.
let module = self.load_module(linker.engine(), &self.module)?;
let host = Host::default();
let mut store = Store::new(&engine, host);
populate_with_wasi( populate_with_wasi(
&mut store,
&mut linker, &mut linker,
&mut store,
module.clone(),
preopen_dirs, preopen_dirs,
&argv, &argv,
&self.vars, &self.vars,
@@ -192,6 +194,12 @@ impl RunCommand {
preopen_sockets, preopen_sockets,
)?; )?;
// If fuel has been configured, we want to add the configured
// fuel amount to this store.
if let Some(fuel) = self.common.fuel {
store.add_fuel(fuel)?;
}
// Load the preload wasm modules. // Load the preload wasm modules.
for (name, path) in self.preloads.iter() { for (name, path) in self.preloads.iter() {
// Read the wasm module binary either as `*.wat` or a raw binary // Read the wasm module binary either as `*.wat` or a raw binary
@@ -207,43 +215,15 @@ impl RunCommand {
// Load the main wasm module. // Load the main wasm module.
match self match self
.load_main_module(&mut store, &mut linker) .load_main_module(&mut store, &mut linker, module)
.with_context(|| format!("failed to run main module `{}`", self.module.display())) .with_context(|| format!("failed to run main module `{}`", self.module.display()))
{ {
Ok(()) => (), Ok(()) => (),
Err(e) => { Err(e) => {
// If a specific WASI error code was requested then that's // Exit the process if Wasmtime understands the error;
// forwarded through to the process here without printing any // otherwise, fall back on Rust's default error printing/return
// extra error information.
if let Some(exit) = e.downcast_ref::<I32Exit>() {
// Print the error message in the usual way.
// On Windows, exit status 3 indicates an abort (see below),
// so return 1 indicating a non-zero status to avoid ambiguity.
if cfg!(windows) && exit.0 >= 3 {
process::exit(1);
}
process::exit(exit.0);
}
// If the program exited because of a trap, return an error code
// to the outside environment indicating a more severe problem
// than a simple failure.
if e.is::<Trap>() {
eprintln!("Error: {:?}", e);
if cfg!(unix) {
// On Unix, return the error code of an abort.
process::exit(128 + libc::SIGABRT);
} else if cfg!(windows) {
// On Windows, return 3.
// https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/abort?view=vs-2019
process::exit(3);
}
}
// Otherwise fall back on Rust's default error printing/return
// code. // code.
return Err(e); return Err(maybe_exit_on_error(e));
} }
} }
@@ -309,7 +289,12 @@ impl RunCommand {
result result
} }
fn load_main_module(&self, store: &mut Store<Host>, linker: &mut Linker<Host>) -> Result<()> { fn load_main_module(
&self,
store: &mut Store<Host>,
linker: &mut Linker<Host>,
module: Module,
) -> Result<()> {
if let Some(timeout) = self.wasm_timeout { if let Some(timeout) = self.wasm_timeout {
store.set_epoch_deadline(1); store.set_epoch_deadline(1);
let engine = store.engine().clone(); let engine = store.engine().clone();
@@ -319,8 +304,6 @@ impl RunCommand {
}); });
} }
// Read the wasm module binary either as `*.wat` or a raw binary.
let module = self.load_module(linker.engine(), &self.module)?;
// The main module might be allowed to have unknown imports, which // The main module might be allowed to have unknown imports, which
// should be defined as traps: // should be defined as traps:
if self.trap_unknown_imports { if self.trap_unknown_imports {
@@ -432,19 +415,22 @@ impl RunCommand {
} }
} }
#[derive(Default)] #[derive(Default, Clone)]
struct Host { struct Host {
wasi: Option<wasmtime_wasi::WasiCtx>, wasi: Option<wasmtime_wasi::WasiCtx>,
#[cfg(feature = "wasi-nn")]
wasi_nn: Option<WasiNnCtx>,
#[cfg(feature = "wasi-crypto")] #[cfg(feature = "wasi-crypto")]
wasi_crypto: Option<WasiCryptoCtx>, wasi_crypto: Option<Arc<WasiCryptoCtx>>,
#[cfg(feature = "wasi-nn")]
wasi_nn: Option<Arc<WasiNnCtx>>,
#[cfg(feature = "wasi-threads")]
wasi_threads: Option<Arc<WasiThreadsCtx<Host>>>,
} }
/// Populates the given `Linker` with WASI APIs. /// Populates the given `Linker` with WASI APIs.
fn populate_with_wasi( fn populate_with_wasi(
store: &mut Store<Host>,
linker: &mut Linker<Host>, linker: &mut Linker<Host>,
store: &mut Store<Host>,
module: Module,
preopen_dirs: Vec<(String, Dir)>, preopen_dirs: Vec<(String, Dir)>,
argv: &[String], argv: &[String],
vars: &[(String, String)], vars: &[(String, String)],
@@ -478,18 +464,6 @@ fn populate_with_wasi(
store.data_mut().wasi = Some(builder.build()); store.data_mut().wasi = Some(builder.build());
} }
if wasi_modules.wasi_nn {
#[cfg(not(feature = "wasi-nn"))]
{
bail!("Cannot enable wasi-nn when the binary is not compiled with this feature.");
}
#[cfg(feature = "wasi-nn")]
{
wasmtime_wasi_nn::add_to_linker(linker, |host| host.wasi_nn.as_mut().unwrap())?;
store.data_mut().wasi_nn = Some(WasiNnCtx::new()?);
}
}
if wasi_modules.wasi_crypto { if wasi_modules.wasi_crypto {
#[cfg(not(feature = "wasi-crypto"))] #[cfg(not(feature = "wasi-crypto"))]
{ {
@@ -497,8 +471,55 @@ fn populate_with_wasi(
} }
#[cfg(feature = "wasi-crypto")] #[cfg(feature = "wasi-crypto")]
{ {
wasmtime_wasi_crypto::add_to_linker(linker, |host| host.wasi_crypto.as_mut().unwrap())?; wasmtime_wasi_crypto::add_to_linker(linker, |host| {
store.data_mut().wasi_crypto = Some(WasiCryptoCtx::new()); // This WASI proposal is currently not protected against
// concurrent access--i.e., when wasi-threads is actively
// spawning new threads, we cannot (yet) safely allow access and
// fail if more than one thread has `Arc`-references to the
// context. Once this proposal is updated (as wasi-common has
// been) to allow concurrent access, this `Arc::get_mut`
// limitation can be removed.
Arc::get_mut(host.wasi_crypto.as_mut().unwrap())
.expect("wasi-crypto is not implemented with multi-threading support")
})?;
store.data_mut().wasi_crypto = Some(Arc::new(WasiCryptoCtx::new()));
}
}
if wasi_modules.wasi_nn {
#[cfg(not(feature = "wasi-nn"))]
{
bail!("Cannot enable wasi-nn when the binary is not compiled with this feature.");
}
#[cfg(feature = "wasi-nn")]
{
wasmtime_wasi_nn::add_to_linker(linker, |host| {
// See documentation for wasi-crypto for why this is needed.
Arc::get_mut(host.wasi_nn.as_mut().unwrap())
.expect("wasi-nn is not implemented with multi-threading support")
})?;
store.data_mut().wasi_nn = Some(Arc::new(WasiNnCtx::new()?));
}
}
if wasi_modules.wasi_threads {
#[cfg(not(feature = "wasi-threads"))]
{
// Silence the unused warning for `module` as it is only used in the
// conditionally-compiled wasi-threads.
drop(&module);
bail!("Cannot enable wasi-threads when the binary is not compiled with this feature.");
}
#[cfg(feature = "wasi-threads")]
{
wasmtime_wasi_threads::add_to_linker(linker, store, &module, |host| {
host.wasi_threads.as_ref().unwrap()
})?;
store.data_mut().wasi_threads = Some(Arc::new(WasiThreadsCtx::new(
module,
Arc::new(linker.clone()),
)?));
} }
} }

25
tests/all/cli_tests.rs
View File

@@ -473,3 +473,28 @@ fn run_cwasm_from_stdin() -> Result<()> {
} }
Ok(()) Ok(())
} }
#[cfg(feature = "wasi-threads")]
#[test]
fn run_threads() -> Result<()> {
let wasm = build_wasm("tests/all/cli_tests/threads.wat")?;
let stdout = run_wasmtime(&[
"run",
"--wasi-modules",
"experimental-wasi-threads",
"--wasm-features",
"threads",
"--disable-cache",
wasm.path().to_str().unwrap(),
])?;
assert!(
stdout
== "Called _start\n\
Running wasi_thread_start\n\
Running wasi_thread_start\n\
Running wasi_thread_start\n\
Done\n"
);
Ok(())
}

62
tests/all/cli_tests/threads.wat Normal file
View File

@@ -0,0 +1,62 @@
(module
;; As we have discussed, it makes sense to make the shared memory an import
;; so that all
(import "" "memory" (memory $shmem 1 1 shared))
(import "wasi_snapshot_preview1" "fd_write"
(func $__wasi_fd_write (param i32 i32 i32 i32) (result i32)))
(import "wasi_snapshot_preview1" "proc_exit"
(func $__wasi_proc_exit (param i32)))
(import "wasi" "thread_spawn"
(func $__wasi_thread_spawn (param i32) (result i32)))
(func (export "_start")
(local $i i32)
;; Print "Called _start".
(call $print (i32.const 32) (i32.const 14))
;; Print "Running wasi_thread_start" in several threads.
(drop (call $__wasi_thread_spawn (i32.const 0)))
(drop (call $__wasi_thread_spawn (i32.const 0)))
(drop (call $__wasi_thread_spawn (i32.const 0)))
;; Wait for all the threads to notify us that they are done.
(loop $again
;; Retrieve the i32 at address 128, compare it to -1 (it should always
;; fail) and load it atomically to check if all three threads are
;; complete. This wait is for 1ms or until notified, whichever is first.
(drop (memory.atomic.wait32 (i32.const 128) (i32.const -1) (i64.const 1000000)))
(br_if $again (i32.lt_s (i32.atomic.load (i32.const 128)) (i32.const 3)))
)
;; Print "Done".
(call $print (i32.const 64) (i32.const 5))
)
;; A threads-enabled module must export this spec-designated entry point.
(func (export "wasi_thread_start") (param $tid i32) (param $start_arg i32)
(call $print (i32.const 96) (i32.const 26))
;; After printing, we atomically increment the value at address 128 and then
;; wake up the main thread's join loop.
(drop (i32.atomic.rmw.add (i32.const 128) (i32.const 1)))
(drop (memory.atomic.notify (i32.const 128) (i32.const 1)))
)
;; A helper function for printing ptr-len strings.
(func $print (param $ptr i32) (param $len i32)
(i32.store (i32.const 8) (local.get $len))
(i32.store (i32.const 4) (local.get $ptr))
(drop (call $__wasi_fd_write
(i32.const 1)
(i32.const 4)
(i32.const 1)
(i32.const 0)))
)
;; We still need to export the shared memory for Wiggle's sake.
(export "memory" (memory $shmem))
(data (i32.const 32) "Called _start\0a")
(data (i32.const 64) "Done\0a")
(data (i32.const 96) "Running wasi_thread_start\0a")
)