T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add epoch-based interruption for cooperative async timeslicing.

Browse Source 
This PR introduces a new way of performing cooperative timeslicing that
is intended to replace the "fuel" mechanism. The tradeoff is that this
mechanism interrupts with less precision: not at deterministic points
where fuel runs out, but rather when the Engine enters a new epoch. The
generated code instrumentation is substantially faster, however, because
it does not need to do as much work as when tracking fuel; it only loads
the global "epoch counter" and does a compare-and-branch at backedges
and function prologues.

This change has been measured as ~twice as fast as fuel-based
timeslicing for some workloads, especially control-flow-intensive
workloads such as the SpiderMonkey JS interpreter on Wasm/WASI.

The intended interface is that the embedder of the `Engine` performs an
`engine.increment_epoch()` call periodically, e.g. once per millisecond.
An async invocation of a Wasm guest on a `Store` can specify a number of
epoch-ticks that are allowed before an async yield back to the
executor's event loop. (The initial amount and automatic "refills" are
configured on the `Store`, just as for fuel.) This call does only
signal-safe work (it increments an `AtomicU64`) so could be invoked from
a periodic signal, or from a thread that wakes up once per period.
This commit is contained in:
Chris Fallin
2022-01-18 17:23:09 -08:00
parent ae476fde60
commit 8a55b5c563
19 changed files with 1034 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

85
crates/wasmtime/src/config.rs
View File

@@ -316,6 +316,91 @@ impl Config {
self
}
/// Enables epoch-based interruption.
///
/// When executing code in async mode, we sometimes want to
/// implement a form of cooperative timeslicing: long-running Wasm
/// guest code should periodically yield to the executor
/// loop. This yielding could be implemented by using "fuel" (see
/// [`consume_fuel`](Config::consume_fuel)). However, fuel
/// instrumentation is somewhat expensive: it modifies the
/// compiled form of the Wasm code so that it maintains a precise
/// instruction count, frequently checking this count against the
/// remaining fuel. If one does not need this precise count or
/// deterministic interruptions, and only needs a periodic
/// interrupt of some form, then It would be better to have a more
/// lightweight mechanism.
///
/// Epoch-based interruption is that mechanism. There is a global
/// "epoch", which is a counter that divides time into arbitrary
/// periods (or epochs). This counter lives on the
/// [`Engine`](crate::Engine) and can be incremented by calling
/// [`Engine::increment_epoch`](crate::Engine::increment_epoch).
/// Epoch-based instrumentation works by setting a "deadline
/// epoch". The compiled code knows the deadline, and at certain
/// points, checks the current epoch against that deadline. It
/// will yield if the deadline has been reached.
///
/// The idea is that checking an infrequently-changing counter is
/// cheaper than counting and frequently storing a precise metric
/// (instructions executed) locally. The interruptions are not
/// deterministic, but if the embedder increments the epoch in a
/// periodic way (say, every regular timer tick by a thread or
/// signal handler), then we can ensure that all async code will
/// yield to the executor within a bounded time.
///
/// The [`Store`](crate::Store) tracks the deadline, and controls
/// what happens when the deadline is reached during
/// execution. Two behaviors are possible:
///
/// - Trap if code is executing when the epoch deadline is
/// met. See
/// [`Store::epoch_deadline_trap`](crate::Store::epoch_deadline_trap).
///
/// - Yield to the executor loop, then resume when the future is
/// next polled. See
/// [`Store::epoch_dealdine_async_yield_and_update`](crate::Store::epoch_deadline_async_yield_and_update).
///
/// The first is the default; set the second for the timeslicing
/// behavior described above.
///
/// This feature is available with or without async
/// support. However, without async support, only the trapping
/// behavior is available. In this mode, epoch-based interruption
/// can serve as a simple external-interruption mechanism.
///
/// An initial deadline can be set before executing code by
/// calling
/// [`Store::set_epoch_deadline`](crate::Store::set_epoch_deadline).
///
/// ## When to use fuel vs. epochs
///
/// In general, epoch-based interruption results in faster
/// execution. This difference is sometimes significant: in some
/// measurements, up to 2-3x. This is because epoch-based
/// interruption does less work: it only watches for a global
/// rarely-changing counter to increment, rather than keeping a
/// local frequently-changing counter and comparing it to a
/// deadline.
///
/// Fuel, in contrast, should be used when *deterministic*
/// yielding or trapping is needed. For example, if it is required
/// that the same function call with the same starting state will
/// always either complete or trap with an out-of-fuel error,
/// deterministically, then fuel with a fixed bound should be
/// used.
///
/// # See Also
///
/// - [`Engine::increment_epoch`](crate::Engine::increment_epoch)
/// - [`Store::set_epoch_deadline`](crate::Store::set_epoch_deadline)
/// - [`Store::epoch_deadline_trap`](crate::Store::epoch_deadline_trap)
/// - [`Store::epoch_deadline_async_yield_and_update`](crate::Store::epoch_deadline_async_yield_and_update)
pub fn epoch_interruption(&mut self, enable: bool) -> &mut Self {
self.tunables.epoch_interruption = enable;
self
}
/// Configures the maximum amount of stack space available for
/// executing WebAssembly code.
///

34
crates/wasmtime/src/engine.rs
View File

@@ -3,6 +3,7 @@ use crate::{Config, Trap};
use anyhow::Result;
#[cfg(feature = "parallel-compilation")]
use rayon::prelude::*;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
#[cfg(feature = "cache")]
use wasmtime_cache::CacheConfig;
@@ -41,6 +42,7 @@ struct EngineInner {
compiler: Box<dyn wasmtime_environ::Compiler>,
allocator: Box<dyn InstanceAllocator>,
signatures: SignatureRegistry,
epoch: AtomicU64,
}
impl Engine {
@@ -65,6 +67,7 @@ impl Engine {
config,
allocator,
signatures: registry,
epoch: AtomicU64::new(0),
}),
})
}
@@ -119,6 +122,37 @@ impl Engine {
&self.inner.signatures
}
pub(crate) fn epoch_counter(&self) -> &AtomicU64 {
&self.inner.epoch
}
pub(crate) fn current_epoch(&self) -> u64 {
self.epoch_counter().load(Ordering::Relaxed)
}
/// Increments the epoch.
///
/// When using epoch-based interruption, currently-executing Wasm
/// code within this engine will trap or yield "soon" when the
/// epoch deadline is reached or exceeded. (The configuration, and
/// the deadline, are set on the `Store`.) The intent of the
/// design is for this method to be called by the embedder at some
/// regular cadence, for example by a thread that wakes up at some
/// interval, or by a signal handler.
///
/// See [`Config::epoch_interruption`](crate::Config::epoch_interruption)
/// for an introduction to epoch-based interruption and pointers
/// to the other relevant methods.
///
/// ## Signal Safety
///
/// This method is signal-safe: it does not make any syscalls, and
/// performs only an atomic increment to the epoch value in
/// memory.
pub fn increment_epoch(&self) {
self.inner.epoch.fetch_add(1, Ordering::Relaxed);
}
/// Ahead-of-time (AOT) compiles a WebAssembly module.
///
/// The `bytes` provided must be in one of two formats:

6
crates/wasmtime/src/module/serialization.rs
View File

@@ -596,6 +596,7 @@ impl<'a> SerializedModule<'a> {
parse_wasm_debuginfo,
interruptable,
consume_fuel,
epoch_interruption,
static_memory_bound_is_maximum,
guard_before_linear_memory,
@@ -636,6 +637,11 @@ impl<'a> SerializedModule<'a> {
)?;
Self::check_bool(interruptable, other.interruptable, "interruption support")?;
Self::check_bool(consume_fuel, other.consume_fuel, "fuel support")?;
Self::check_bool(
epoch_interruption,
other.epoch_interruption,
"epoch interruption",
)?;
Self::check_bool(
static_memory_bound_is_maximum,
other.static_memory_bound_is_maximum,

227
crates/wasmtime/src/store.rs
View File

@@ -88,6 +88,7 @@ use std::mem::{self, ManuallyDrop};
use std::ops::{Deref, DerefMut};
use std::pin::Pin;
use std::ptr;
use std::sync::atomic::AtomicU64;
use std::sync::Arc;
use std::task::{Context, Poll};
use wasmtime_runtime::{
@@ -272,6 +273,7 @@ pub struct StoreOpaque {
#[cfg(feature = "async")]
async_state: AsyncState,
out_of_gas_behavior: OutOfGas,
epoch_deadline_behavior: EpochDeadline,
store_data: StoreData,
default_callee: InstanceHandle,
@@ -379,6 +381,18 @@ enum OutOfGas {
},
}
/// What to do when the engine epoch reaches the deadline for a Store
/// during execution of a function using that store.
#[derive(Copy, Clone)]
enum EpochDeadline {
/// Return early with a trap.
Trap,
/// Extend the deadline by the specified number of ticks after
/// yielding to the async executor loop.
#[cfg(feature = "async")]
YieldAndExtendDeadline { delta: u64 },
}
impl<T> Store<T> {
/// Creates a new [`Store`] to be associated with the given [`Engine`] and
/// `data` provided.
@@ -435,6 +449,7 @@ impl<T> Store<T> {
current_poll_cx: UnsafeCell::new(ptr::null_mut()),
},
out_of_gas_behavior: OutOfGas::Trap,
epoch_deadline_behavior: EpochDeadline::Trap,
store_data: StoreData::new(),
default_callee,
hostcall_val_storage: Vec::new(),
@@ -809,6 +824,86 @@ impl<T> Store<T> {
self.inner
.out_of_fuel_async_yield(injection_count, fuel_to_inject)
}
/// Sets the epoch deadline to a certain number of ticks in the future.
///
/// When the Wasm guest code is compiled with epoch-interruption
/// instrumentation
/// ([`Config::epoch_interruption()`](crate::Config::epoch_interruption)),
/// and when the `Engine`'s epoch is incremented
/// ([`Engine::increment_epoch()`](crate::Engine::increment_epoch))
/// past a deadline, execution can be configured to either trap or
/// yield and then continue.
///
/// This deadline is always set relative to the current epoch:
/// `delta_beyond_current` ticks in the future. The deadline can
/// be set explicitly via this method, or refilled automatically
/// on a yield if configured via
/// [`epoch_deadline_async_yield_and_update()`](Store::epoch_deadline_async_yield_and_update). After
/// this method is invoked, the deadline is reached when
/// [`Engine::increment_epoch()`] has been invoked at least
/// `ticks_beyond_current` times.
///
/// See documentation on
/// [`Config::epoch_interruption()`](crate::Config::epoch_interruption)
/// for an introduction to epoch-based interruption.
pub fn set_epoch_deadline(&mut self, ticks_beyond_current: u64) {
self.inner.set_epoch_deadline(ticks_beyond_current);
}
/// Configures epoch-deadline expiration to trap.
///
/// When epoch-interruption-instrumented code is executed on this
/// store and the epoch deadline is reached before completion,
/// with the store configured in this way, execution will
/// terminate with a trap as soon as an epoch check in the
/// instrumented code is reached.
///
/// This behavior is the default if the store is not otherwise
/// configured via
/// [`epoch_deadline_trap()`](Store::epoch_deadline_trap) or
/// [`epoch_deadline_async_yield_and_update()`](Store::epoch_deadline_async_yield_and_update).
///
/// This setting is intended to allow for coarse-grained
/// interruption, but not a deterministic deadline of a fixed,
/// finite interval. For deterministic interruption, see the
/// "fuel" mechanism instead.
///
/// See documentation on
/// [`Config::epoch_interruption()`](crate::Config::epoch_interruption)
/// for an introduction to epoch-based interruption.
pub fn epoch_deadline_trap(&mut self) {
self.inner.epoch_deadline_trap();
}
#[cfg_attr(nightlydoc, doc(cfg(feature = "async")))]
/// Configures epoch-deadline expiration to yield to the async
/// caller and the update the deadline.
///
/// When epoch-interruption-instrumented code is executed on this
/// store and the epoch deadline is reached before completion,
/// with the store configured in this way, execution will yield
/// (the future will return `Pending` but re-awake itself for
/// later execution) and, upon resuming, the store will be
/// configured with an epoch deadline equal to the current epoch
/// plus `delta` ticks.
///
/// This setting is intended to allow for cooperative timeslicing
/// of multiple CPU-bound Wasm guests in different stores, all
/// executing under the control of an async executor. To drive
/// this, stores should be configured to "yield and update"
/// automatically with this function, and some external driver (a
/// thread that wakes up periodically, or a timer
/// signal/interrupt) should call
/// [`Engine::increment_epoch()`](crate::Engine::increment_epoch).
///
/// See documentation on
/// [`Config::epoch_interruption()`](crate::Config::epoch_interruption)
/// for an introduction to epoch-based interruption.
#[cfg(feature = "async")]
pub fn epoch_deadline_async_yield_and_update(&mut self, delta: u64) {
self.inner.epoch_deadline_async_yield_and_update(delta);
}
}
impl<'a, T> StoreContext<'a, T> {
@@ -913,6 +1008,31 @@ impl<'a, T> StoreContextMut<'a, T> {
self.0
.out_of_fuel_async_yield(injection_count, fuel_to_inject)
}
/// Sets the epoch deadline to a certain number of ticks in the future.
///
/// For more information see [`Store::set_epoch_deadline`].
pub fn set_epoch_deadline(&mut self, ticks_beyond_current: u64) {
self.0.set_epoch_deadline(ticks_beyond_current);
}
/// Configures epoch-deadline expiration to trap.
///
/// For more information see [`Store::epoch_deadline_trap`].
pub fn epoch_deadline_trap(&mut self) {
self.0.epoch_deadline_trap();
}
#[cfg_attr(nightlydoc, doc(cfg(feature = "async")))]
/// Configures epoch-deadline expiration to yield to the async
/// caller and the update the deadline.
///
/// For more information see
/// [`Store::epoch_deadline_async_yield_and_update`].
#[cfg(feature = "async")]
pub fn epoch_deadline_async_yield_and_update(&mut self, delta: u64) {
self.0.epoch_deadline_async_yield_and_update(delta);
}
}
impl<T> StoreInner<T> {
@@ -1093,13 +1213,12 @@ impl StoreOpaque {
};
}
/// Yields execution to the caller on out-of-gas
/// Yields execution to the caller on out-of-gas or epoch interruption.
///
/// This only works on async futures and stores, and assumes that we're
/// executing on a fiber. This will yield execution back to the caller once
/// and when we come back we'll continue with `fuel_to_inject` more fuel.
/// executing on a fiber. This will yield execution back to the caller once.
#[cfg(feature = "async")]
fn out_of_gas_yield(&mut self, fuel_to_inject: u64) -> Result<(), Trap> {
fn async_yield_impl(&mut self) -> Result<(), Trap> {
// Small future that yields once and then returns ()
#[derive(Default)]
struct Yield {
@@ -1124,19 +1243,15 @@ impl StoreOpaque {
}
let mut future = Yield::default();
let result = unsafe { self.async_cx().block_on(Pin::new_unchecked(&mut future)) };
match result {
// If this finished successfully then we were resumed normally via a
// `poll`, so inject some more fuel and keep going.
Ok(()) => {
self.add_fuel(fuel_to_inject).unwrap();
Ok(())
}
// If the future was dropped while we were yielded, then we need to
// clean up this fiber. Do so by raising a trap which will abort all
// wasm and get caught on the other side to clean things up.
Err(trap) => Err(trap),
}
// When control returns, we have a `Result<(), Trap>` passed
// in from the host fiber. If this finished successfully then
// we were resumed normally via a `poll`, so keep going. If
// the future was dropped while we were yielded, then we need
// to clean up this fiber. Do so by raising a trap which will
// abort all wasm and get caught on the other side to clean
// things up.
unsafe { self.async_cx().block_on(Pin::new_unchecked(&mut future)) }
}
fn add_fuel(&mut self, fuel: u64) -> Result<()> {
@@ -1187,6 +1302,22 @@ impl StoreOpaque {
}
}
fn epoch_deadline_trap(&mut self) {
self.epoch_deadline_behavior = EpochDeadline::Trap;
}
fn epoch_deadline_async_yield_and_update(&mut self, delta: u64) {
assert!(
self.async_support(),
"cannot use `epoch_deadline_async_yield_and_update` without enabling async support in the config"
);
#[cfg(feature = "async")]
{
self.epoch_deadline_behavior = EpochDeadline::YieldAndExtendDeadline { delta };
}
drop(delta); // suppress warning in non-async build
}
#[inline]
pub fn signal_handler(&self) -> Option<*const SignalHandler<'static>> {
let handler = self.signal_handler.as_ref()?;
@@ -1535,6 +1666,10 @@ unsafe impl<T> wasmtime_runtime::Store for StoreInner<T> {
<StoreOpaque>::vminterrupts(self)
}
fn epoch_ptr(&self) -> *const AtomicU64 {
self.engine.epoch_counter() as *const _
}
fn externref_activations_table(
&mut self,
) -> (
@@ -1651,7 +1786,10 @@ unsafe impl<T> wasmtime_runtime::Store for StoreInner<T> {
}
*injection_count -= 1;
let fuel = *fuel_to_inject;
self.out_of_gas_yield(fuel)?;
self.async_yield_impl()?;
if fuel > 0 {
self.add_fuel(fuel).unwrap();
}
Ok(())
}
#[cfg(not(feature = "async"))]
@@ -1669,6 +1807,59 @@ unsafe impl<T> wasmtime_runtime::Store for StoreInner<T> {
impl std::error::Error for OutOfGasError {}
}
fn new_epoch(&mut self) -> Result<u64, anyhow::Error> {
return match &self.epoch_deadline_behavior {
&EpochDeadline::Trap => Err(anyhow::Error::new(EpochDeadlineError)),
#[cfg(feature = "async")]
&EpochDeadline::YieldAndExtendDeadline { delta } => {
// Do the async yield. May return a trap if future was
// canceled while we're yielded.
self.async_yield_impl()?;
// Set a new deadline.
self.set_epoch_deadline(delta);
// Return the new epoch deadline so the Wasm code
// doesn't have to reload it.
Ok(self.get_epoch_deadline())
}
};
#[derive(Debug)]
struct EpochDeadlineError;
impl fmt::Display for EpochDeadlineError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("epoch deadline reached during execution")
}
}
impl std::error::Error for EpochDeadlineError {}
}
}
impl<T> StoreInner<T> {
pub(crate) fn set_epoch_deadline(&mut self, delta: u64) {
// Set a new deadline based on the "epoch deadline delta".
//
// Safety: this is safe because the epoch deadline in the
// `VMInterrupts` is accessed only here and by Wasm guest code
// running in this store, and we have a `&mut self` here.
//
// Also, note that when this update is performed while Wasm is
// on the stack, the Wasm will reload the new value once we
// return into it.
let epoch_deadline = unsafe { (*self.vminterrupts()).epoch_deadline.get_mut() };
*epoch_deadline = self.engine().current_epoch() + delta;
}
fn get_epoch_deadline(&self) -> u64 {
// Safety: this is safe because, as above, it is only invoked
// from within `new_epoch` which is called from guest Wasm
// code, which will have an exclusive borrow on the Store.
let epoch_deadline = unsafe { (*self.vminterrupts()).epoch_deadline.get_mut() };
*epoch_deadline
}
}
impl<T: Default> Default for Store<T> {