* Add shared memories This change adds the ability to use shared memories in Wasmtime when the [threads proposal] is enabled. Shared memories are annotated as `shared` in the WebAssembly syntax, e.g., `(memory 1 1 shared)`, and are protected from concurrent access during `memory.size` and `memory.grow`. [threads proposal]: https://github.com/WebAssembly/threads/blob/master/proposals/threads/Overview.md In order to implement this in Wasmtime, there are two main cases to cover: - a program may simply create a shared memory and possibly export it; this means that Wasmtime itself must be able to create shared memories - a user may create a shared memory externally and pass it in as an import during instantiation; this is the case when the program contains code like `(import "env" "memory" (memory 1 1 shared))`--this case is handled by a new Wasmtime API type--`SharedMemory` Because of the first case, this change allows any of the current memory-creation mechanisms to work as-is. Wasmtime can still create either static or dynamic memories in either on-demand or pooling modes, and any of these memories can be considered shared. When shared, the `Memory` runtime container will lock appropriately during `memory.size` and `memory.grow` operations; since all memories use this container, it is an ideal place for implementing the locking once and once only. The second case is covered by the new `SharedMemory` structure. It uses the same `Mmap` allocation under the hood as non-shared memories, but allows the user to perform the allocation externally to Wasmtime and share the memory across threads (via an `Arc`). The pointer address to the actual memory is carefully wired through and owned by the `SharedMemory` structure itself. This means that there are differing views of where to access the pointer (i.e., `VMMemoryDefinition`): for owned memories (the default), the `VMMemoryDefinition` is stored directly by the `VMContext`; in the `SharedMemory` case, however, this `VMContext` must point to this separate structure. To ensure that the `VMContext` can always point to the correct `VMMemoryDefinition`, this change alters the `VMContext` structure. Since a `SharedMemory` owns its own `VMMemoryDefinition`, the `defined_memories` table in the `VMContext` becomes a sequence of pointers--in the shared memory case, they point to the `VMMemoryDefinition` owned by the `SharedMemory` and in the owned memory case (i.e., not shared) they point to `VMMemoryDefinition`s stored in a new table, `owned_memories`. This change adds an additional indirection (through the `*mut VMMemoryDefinition` pointer) that could add overhead. Using an imported memory as a proxy, we measured a 1-3% overhead of this approach on the `pulldown-cmark` benchmark. To avoid this, Cranelift-generated code will special-case the owned memory access (i.e., load a pointer directly to the `owned_memories` entry) for `memory.size` so that only shared memories (and imported memories, as before) incur the indirection cost. * review: remove thread feature check * review: swap wasmtime-types dependency for existing wasmtime-environ use * review: remove unused VMMemoryUnion * review: reword cross-engine error message * review: improve tests * review: refactor to separate prevent Memory <-> SharedMemory conversion * review: into_shared_memory -> as_shared_memory * review: remove commented out code * review: limit shared min/max to 32 bits * review: skip imported memories * review: imported memories are not owned * review: remove TODO * review: document unsafe send + sync * review: add limiter assertion * review: remove TODO * review: improve tests * review: fix doc test * fix: fixes based on discussion with Alex This changes several key parts: - adds memory indexes to imports and exports - makes `VMMemoryDefinition::current_length` an atomic usize * review: add `Extern::SharedMemory` * review: remove TODO * review: atomically load from VMMemoryDescription in JIT-generated code * review: add test probing the last available memory slot across threads * fix: move assertion to new location due to rebase * fix: doc link * fix: add TODOs to c-api * fix: broken doc link * fix: modify pooling allocator messages in tests * review: make owned_memory_index panic instead of returning an option * review: clarify calculation of num_owned_memories * review: move 'use' to top of file * review: change '*const [u8]' to '*mut [u8]' * review: remove TODO * review: avoid hard-coding memory index * review: remove 'preallocation' parameter from 'Memory::_new' * fix: component model memory length * review: check that shared memory plans are static * review: ignore growth limits for shared memory * review: improve atomic store comment * review: add FIXME for memory growth failure * review: add comment about absence of bounds-checked 'memory.size' * review: make 'current_length()' doc comment more precise * review: more comments related to memory.size non-determinism * review: make 'vmmemory' unreachable for shared memory * review: move code around * review: thread plan through to 'wrap()' * review: disallow shared memory allocation with the pooling allocator
140 lines
4.1 KiB
Rust
140 lines
4.1 KiB
Rust
use crate::{
|
|
wasm_externkind_t, wasm_externtype_t, wasm_func_t, wasm_global_t, wasm_memory_t, wasm_table_t,
|
|
CStoreContext, StoreRef,
|
|
};
|
|
use std::mem::ManuallyDrop;
|
|
use wasmtime::{Extern, Func, Global, Memory, Table};
|
|
|
|
#[derive(Clone)]
|
|
pub struct wasm_extern_t {
|
|
pub(crate) store: StoreRef,
|
|
pub(crate) which: Extern,
|
|
}
|
|
|
|
wasmtime_c_api_macros::declare_ref!(wasm_extern_t);
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn wasm_extern_kind(e: &wasm_extern_t) -> wasm_externkind_t {
|
|
match e.which {
|
|
Extern::Func(_) => crate::WASM_EXTERN_FUNC,
|
|
Extern::Global(_) => crate::WASM_EXTERN_GLOBAL,
|
|
Extern::Table(_) => crate::WASM_EXTERN_TABLE,
|
|
Extern::Memory(_) => crate::WASM_EXTERN_MEMORY,
|
|
Extern::SharedMemory(_) => todo!(),
|
|
}
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn wasm_extern_type(e: &wasm_extern_t) -> Box<wasm_externtype_t> {
|
|
Box::new(wasm_externtype_t::new(e.which.ty(&e.store.context())))
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn wasm_extern_as_func(e: &wasm_extern_t) -> Option<&wasm_func_t> {
|
|
wasm_func_t::try_from(e)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn wasm_extern_as_func_const(e: &wasm_extern_t) -> Option<&wasm_func_t> {
|
|
wasm_extern_as_func(e)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn wasm_extern_as_global(e: &wasm_extern_t) -> Option<&wasm_global_t> {
|
|
wasm_global_t::try_from(e)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn wasm_extern_as_global_const(e: &wasm_extern_t) -> Option<&wasm_global_t> {
|
|
wasm_extern_as_global(e)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn wasm_extern_as_table(e: &wasm_extern_t) -> Option<&wasm_table_t> {
|
|
wasm_table_t::try_from(e)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn wasm_extern_as_table_const(e: &wasm_extern_t) -> Option<&wasm_table_t> {
|
|
wasm_extern_as_table(e)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn wasm_extern_as_memory(e: &wasm_extern_t) -> Option<&wasm_memory_t> {
|
|
wasm_memory_t::try_from(e)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn wasm_extern_as_memory_const(e: &wasm_extern_t) -> Option<&wasm_memory_t> {
|
|
wasm_extern_as_memory(e)
|
|
}
|
|
|
|
#[repr(C)]
|
|
pub struct wasmtime_extern_t {
|
|
pub kind: wasmtime_extern_kind_t,
|
|
pub of: wasmtime_extern_union,
|
|
}
|
|
|
|
pub type wasmtime_extern_kind_t = u8;
|
|
pub const WASMTIME_EXTERN_FUNC: wasmtime_extern_kind_t = 0;
|
|
pub const WASMTIME_EXTERN_GLOBAL: wasmtime_extern_kind_t = 1;
|
|
pub const WASMTIME_EXTERN_TABLE: wasmtime_extern_kind_t = 2;
|
|
pub const WASMTIME_EXTERN_MEMORY: wasmtime_extern_kind_t = 3;
|
|
|
|
#[repr(C)]
|
|
pub union wasmtime_extern_union {
|
|
pub func: Func,
|
|
pub table: Table,
|
|
pub global: Global,
|
|
pub memory: Memory,
|
|
}
|
|
|
|
impl wasmtime_extern_t {
|
|
pub unsafe fn to_extern(&self) -> Extern {
|
|
match self.kind {
|
|
WASMTIME_EXTERN_FUNC => Extern::Func(self.of.func),
|
|
WASMTIME_EXTERN_GLOBAL => Extern::Global(self.of.global),
|
|
WASMTIME_EXTERN_TABLE => Extern::Table(self.of.table),
|
|
WASMTIME_EXTERN_MEMORY => Extern::Memory(self.of.memory),
|
|
other => panic!("unknown wasm_extern_kind_t: {}", other),
|
|
}
|
|
}
|
|
}
|
|
|
|
impl From<Extern> for wasmtime_extern_t {
|
|
fn from(item: Extern) -> wasmtime_extern_t {
|
|
match item {
|
|
Extern::Func(func) => wasmtime_extern_t {
|
|
kind: WASMTIME_EXTERN_FUNC,
|
|
of: wasmtime_extern_union { func },
|
|
},
|
|
Extern::Global(global) => wasmtime_extern_t {
|
|
kind: WASMTIME_EXTERN_GLOBAL,
|
|
of: wasmtime_extern_union { global },
|
|
},
|
|
Extern::Table(table) => wasmtime_extern_t {
|
|
kind: WASMTIME_EXTERN_TABLE,
|
|
of: wasmtime_extern_union { table },
|
|
},
|
|
Extern::Memory(memory) => wasmtime_extern_t {
|
|
kind: WASMTIME_EXTERN_MEMORY,
|
|
of: wasmtime_extern_union { memory },
|
|
},
|
|
Extern::SharedMemory(_memory) => todo!(),
|
|
}
|
|
}
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn wasmtime_extern_delete(e: &mut ManuallyDrop<wasmtime_extern_t>) {
|
|
ManuallyDrop::drop(e);
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn wasmtime_extern_type(
|
|
store: CStoreContext<'_>,
|
|
e: &wasmtime_extern_t,
|
|
) -> Box<wasm_externtype_t> {
|
|
Box::new(wasm_externtype_t::new(e.to_extern().ty(store)))
|
|
}
|