T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Implement support for async functions in Wasmtime (#2434)

Browse Source 
* Implement support for `async` functions in Wasmtime

This is an implementation of [RFC 2] in Wasmtime which is to support
`async`-defined host functions. At a high level support is added by
executing WebAssembly code that might invoke an asynchronous host
function on a separate native stack. When the host function's future is
not ready we switch back to the main native stack to continue execution.

There's a whole bunch of details in this commit, and it's a bit much to
go over them all here in this commit message. The most important changes
here are:

* A new `wasmtime-fiber` crate has been written to manage the low-level
  details of stack-switching. Unixes use `mmap` to allocate a stack and
  Windows uses the native fibers implementation. We'll surely want to
  refactor this to move stack allocation elsewhere in the future. Fibers
  are intended to be relatively general with a lot of type paremters to
  fling values back and forth across suspension points. The whole crate
  is a giant wad of `unsafe` unfortunately and involves handwritten
  assembly with custom dwarf CFI directives to boot. Definitely deserves
  a close eye in review!

* The `Store` type has two new methods -- `block_on` and `on_fiber`
  which bridge between the async and non-async worlds. Lots of unsafe
  fiddly bits here as we're trying to communicate context pointers
  between disparate portions of the code. Extra eyes and care in review
  is greatly appreciated.

* The APIs for binding `async` functions are unfortunately pretty ugly
  in `Func`. This is mostly due to language limitations and compiler
  bugs (I believe) in Rust. Instead of `Func::wrap` we have a
  `Func::wrapN_async` family of methods, and we've also got a whole
  bunch of `Func::getN_async` methods now too. It may be worth
  rethinking the API of `Func` to try to make the documentation page
  actually grok'able.

This isn't super heavily tested but the various test should suffice for
engaging hopefully nearly all the infrastructure in one form or another.
This is just the start though!

[RFC 2]: https://github.com/bytecodealliance/rfcs/pull/2

* Add wasmtime-fiber to publish script

* Save vector/float registers on ARM too.

* Fix a typo

* Update lock file

* Implement periodically yielding with fuel consumption

This commit implements APIs on `Store` to periodically yield execution
of futures through the consumption of fuel. When fuel runs out a
future's execution is yielded back to the caller, and then upon
resumption fuel is re-injected. The goal of this is to allow cooperative
multi-tasking with futures.

* Fix compile without async

* Save/restore the frame pointer in fiber switching

Turns out this is another caller-saved register!

* Simplify x86_64 fiber asm

Take a leaf out of aarch64's playbook and don't have extra memory to
load/store these arguments, instead leverage how `wasmtime_fiber_switch`
already loads a bunch of data into registers which we can then
immediately start using on a fiber's start without any extra memory
accesses.

* Add x86 support to wasmtime-fiber

* Add ARM32 support to fiber crate

* Make fiber build file probing more flexible

* Use CreateFiberEx on Windows

* Remove a stray no-longer-used trait declaration

* Don't reach into `Caller` internals

* Tweak async fuel to eventually run out.

With fuel it's probably best to not provide any way to inject infinite
fuel.

* Fix some typos

* Cleanup asm a bit

* Use a shared header file to deduplicate some directives
* Guarantee hidden visibility for functions
* Enable gc-sections on macOS x86_64
* Add `.type` annotations for ARM

* Update lock file

* Fix compile error

* Review comments
This commit is contained in:
Alex Crichton
2021-02-26 16:19:56 -06:00
committed by GitHub
parent 2b3eff8582
commit 7795a230f2
22 changed files with 2418 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files

122
crates/fiber/src/arch/aarch64.S Normal file
View File

@@ -0,0 +1,122 @@
// A WORD OF CAUTION
//
// This entire file basically needs to be kept in sync with itself. It's not
// really possible to modify just one bit of this file without understanding
// all the other bits. Documentation tries to reference various bits here and
// there but try to make sure to read over everything before tweaking things!
//
// Also at this time this file is heavily based off the x86_64 file, so you'll
// probably want to read that one as well.
#include "header.h"
// fn(top_of_stack(%x0): *mut u8)
HIDDEN(wasmtime_fiber_switch)
GLOBL(wasmtime_fiber_switch)
.p2align 2
TYPE(wasmtime_fiber_switch)
FUNCTION(wasmtime_fiber_switch):
// Save all callee-saved registers on the stack since we're assuming
// they're clobbered as a result of the stack switch.
stp lr, fp, [sp, -16]!
stp x20, x19, [sp, -16]!
stp x22, x21, [sp, -16]!
stp x24, x23, [sp, -16]!
stp x26, x25, [sp, -16]!
stp x28, x27, [sp, -16]!
stp d9, d8, [sp, -16]!
stp d11, d10, [sp, -16]!
stp d13, d12, [sp, -16]!
stp d15, d14, [sp, -16]!
// Load our previously saved stack pointer to resume to, and save off our
// current stack pointer on where to come back to eventually.
ldr x8, [x0, -0x10]
mov x9, sp
str x9, [x0, -0x10]
// Switch to the new stack and restore all our callee-saved registers after
// the switch and return to our new stack.
mov sp, x8
ldp d15, d14, [sp], 16
ldp d13, d12, [sp], 16
ldp d11, d10, [sp], 16
ldp d9, d8, [sp], 16
ldp x28, x27, [sp], 16
ldp x26, x25, [sp], 16
ldp x24, x23, [sp], 16
ldp x22, x21, [sp], 16
ldp x20, x19, [sp], 16
ldp lr, fp, [sp], 16
ret
SIZE(wasmtime_fiber_switch)
// fn(
// top_of_stack(%x0): *mut u8,
// entry_point(%x1): extern fn(*mut u8, *mut u8),
// entry_arg0(%x2): *mut u8,
// )
HIDDEN(wasmtime_fiber_init)
GLOBL(wasmtime_fiber_init)
.p2align 2
TYPE(wasmtime_fiber_init)
FUNCTION(wasmtime_fiber_init):
adr x8, FUNCTION(wasmtime_fiber_start)
stp x0, x8, [x0, -0x28] // x0 => x19, x8 => lr
stp x2, x1, [x0, -0x38] // x1 => x20, x2 => x21
// `wasmtime_fiber_switch` has an 0xa0 byte stack, and we add 0x10 more for
// the original reserved 16 bytes.
add x8, x0, -0xb0
str x8, [x0, -0x10]
ret
SIZE(wasmtime_fiber_init)
.p2align 2
TYPE(wasmtime_fiber_start)
FUNCTION(wasmtime_fiber_start):
.cfi_startproc simple
// See the x86_64 file for more commentary on what these CFI directives are
// doing. Like over there note that the relative offsets to registers here
// match the frame layout in `wasmtime_fiber_switch`.
.cfi_escape 0x0f, /* DW_CFA_def_cfa_expression */ \
5, /* the byte length of this expression */ \
0x6f, /* DW_OP_reg31(%sp) */ \
0x06, /* DW_OP_deref */ \
0x23, 0xa0, 0x1 /* DW_OP_plus_uconst 0xa0 */
.cfi_rel_offset lr, -0x10
.cfi_rel_offset x19, -0x18
.cfi_rel_offset x20, -0x20
.cfi_rel_offset x21, -0x28
.cfi_rel_offset x22, -0x30
.cfi_rel_offset x23, -0x38
.cfi_rel_offset x24, -0x40
.cfi_rel_offset x25, -0x48
.cfi_rel_offset x26, -0x50
.cfi_rel_offset x27, -0x58
.cfi_rel_offset x29, -0x60
// Load our two arguments from the stack, where x1 is our start procedure
// and x0 is its first argument. This also blows away the stack space used
// by those two arguments.
mov x0, x21
mov x1, x19
// ... and then we call the function! Note that this is a function call so
// our frame stays on the stack to backtrace through.
blr x20
// .. technically we shouldn't get here, and I would like to write in an
// instruction which just aborts, but I don't know such an instruction in
// aarch64 land.
.cfi_endproc
SIZE(wasmtime_fiber_start)
// This omits the `.subsections_via_symbols` directive on macOS which means we
// can't GC specific intrinsics from this file, but it enables usage of the
// `adr` instruction above in lieu of figuring out a slightly more complicated
// way of implementing that.
#ifndef CFG_TARGET_OS_macos
FOOTER
#endif

83
crates/fiber/src/arch/arm.S Normal file
View File

@@ -0,0 +1,83 @@
// A WORD OF CAUTION
//
// This entire file basically needs to be kept in sync with itself. It's not
// really possible to modify just one bit of this file without understanding
// all the other bits. Documentation tries to reference various bits here and
// there but try to make sure to read over everything before tweaking things!
//
// Also at this time this file is heavily based off the x86_64 file, so you'll
// probably want to read that one as well.
#include "header.h"
// fn(top_of_stack(%r0): *mut u8)
HIDDEN(wasmtime_fiber_switch)
GLOBL(wasmtime_fiber_switch)
TYPE(wasmtime_fiber_switch)
FUNCTION(wasmtime_fiber_switch):
// Save callee-saved registers
push {r4-r11,lr}
// Swap stacks, recording our current stack pointer
ldr r4, [r0, #-0x08]
str sp, [r0, #-0x08]
mov sp, r4
// Restore and return
pop {r4-r11,lr}
bx lr
SIZE(wasmtime_fiber_switch)
// fn(
// top_of_stack(%r0): *mut u8,
// entry_point(%r1): extern fn(*mut u8, *mut u8),
// entry_arg0(%r2): *mut u8,
// )
HIDDEN(wasmtime_fiber_init)
GLOBL(wasmtime_fiber_init)
TYPE(wasmtime_fiber_init)
FUNCTION(wasmtime_fiber_init):
adr r3, FUNCTION(wasmtime_fiber_start)
str r3, [r0, #-0x0c] // => lr
str r0, [r0, #-0x10] // => r11
str r1, [r0, #-0x14] // => r10
str r2, [r0, #-0x18] // => r9
add r3, r0, #-0x2c
str r3, [r0, #-0x08]
bx lr
SIZE(wasmtime_fiber_init)
FUNCTION(wasmtime_fiber_start):
.cfi_startproc simple
// See the x86_64 file for more commentary on what these CFI directives are
// doing. Like over there note that the relative offsets to registers here
// match the frame layout in `wasmtime_fiber_switch`.
//
// TODO: this is only lightly tested. This gets backtraces in gdb but not
// at runtime. Perhaps the libgcc at runtime was too old? Doesn't support
// something here? Unclear. Will need investigation if someone ends up
// needing this and it still doesn't work.
.cfi_escape 0x0f, /* DW_CFA_def_cfa_expression */ \
5, /* the byte length of this expression */ \
0x7d, 0x00, /* DW_OP_breg14(%sp) + 0 */ \
0x06, /* DW_OP_deref */ \
0x23, 0x24 /* DW_OP_plus_uconst 0x24 */
.cfi_rel_offset lr, -0x04
.cfi_rel_offset r11, -0x08
.cfi_rel_offset r10, -0x0c
.cfi_rel_offset r9, -0x10
.cfi_rel_offset r8, -0x14
.cfi_rel_offset r7, -0x18
.cfi_rel_offset r6, -0x1c
.cfi_rel_offset r5, -0x20
.cfi_rel_offset r4, -0x24
mov r1, r11
mov r0, r9
blx r10
.cfi_endproc
SIZE(wasmtime_fiber_start)
FOOTER

36
crates/fiber/src/arch/header.h Normal file
View File

@@ -0,0 +1,36 @@
#ifndef __wasmtime_common_h
#define __wasmtime_common_h
#if CFG_TARGET_OS_macos
.section __TEXT,__text,regular,pure_instructions
#define GLOBL(fnname) .globl _##fnname
#define HIDDEN(fnname) .private_extern _##fnname
#define TYPE(fnname)
#define FUNCTION(fnname) _##fnname
#define SIZE(fnname)
// Tells the linker it's safe to gc symbols away if not used.
#define FOOTER .subsections_via_symbols
#else
.text
#define GLOBL(fnname) .globl fnname
#define HIDDEN(fnname) .hidden fnname
#ifdef CFG_TARGET_ARCH_arm
#define TYPE(fnname) .type fnname,%function
#else
#define TYPE(fnname) .type fnname,@function
#endif
#define FUNCTION(fnname) fnname
#define SIZE(fnname) .size fnname,.-fnname
// Mark that we don't need executable stack.
#define FOOTER .section .note.GNU-stack,"",%progbits
#endif
#endif // __wasmtime_common_h

5
crates/fiber/src/arch/windows.c Normal file
View File

@@ -0,0 +1,5 @@
#include <windows.h>
LPVOID wasmtime_fiber_get_current() {
return GetCurrentFiber();
}

107
crates/fiber/src/arch/x86.S Normal file
View File

@@ -0,0 +1,107 @@
// A WORD OF CAUTION
//
// This entire file basically needs to be kept in sync with itself. It's not
// really possible to modify just one bit of this file without understanding
// all the other bits. Documentation tries to reference various bits here and
// there but try to make sure to read over everything before tweaking things!
//
// This file is modeled after x86_64.S and comments are not copied over. For
// reference be sure to review the other file. Note that the pointer size is
// different so the reserved space at the top of the stack is 8 bytes, not 16
// bytes. Still two pointers though.
#include "header.h"
// fn(top_of_stack: *mut u8)
HIDDEN(wasmtime_fiber_switch)
GLOBL(wasmtime_fiber_switch)
TYPE(wasmtime_fiber_switch)
FUNCTION(wasmtime_fiber_switch):
// Load our stack-to-use
mov 0x4(%esp), %eax
mov -0x8(%eax), %ecx
// Save callee-saved registers
push %ebp
push %ebx
push %esi
push %edi
// Save our current stack and jump to the stack-to-use
mov %esp, -0x8(%eax)
mov %ecx, %esp
// Restore callee-saved registers
pop %edi
pop %esi
pop %ebx
pop %ebp
ret
SIZE(wasmtime_fiber_switch)
// fn(
// top_of_stack: *mut u8,
// entry_point: extern fn(*mut u8, *mut u8),
// entry_arg0: *mut u8,
// )
HIDDEN(wasmtime_fiber_init)
GLOBL(wasmtime_fiber_init)
TYPE(wasmtime_fiber_init)
FUNCTION(wasmtime_fiber_init):
mov 4(%esp), %eax
// move top_of_stack to the 2nd argument
mov %eax, -0x0c(%eax)
// move entry_arg0 to the 1st argument
mov 12(%esp), %ecx
mov %ecx, -0x10(%eax)
// Move our start function to the return address which the `ret` in
// `wasmtime_fiber_start` will return to.
lea FUNCTION(wasmtime_fiber_start), %ecx
mov %ecx, -0x14(%eax)
// And move `entry_point` to get loaded into `%ebp` through the context
// switch. This'll get jumped to in `wasmtime_fiber_start`.
mov 8(%esp), %ecx
mov %ecx, -0x18(%eax)
// Our stack from top-to-bottom looks like:
//
// * 8 bytes of reserved space per unix.rs (two-pointers space)
// * 8 bytes of arguments (two arguments wasmtime_fiber_start forwards)
// * 4 bytes of return address
// * 16 bytes of saved registers
//
// Note that after the return address the stack is conveniently 16-byte
// aligned as required, so we just leave the arguments on the stack in
// `wasmtime_fiber_start` and immediately do the call.
lea -0x24(%eax), %ecx
mov %ecx, -0x08(%eax)
ret
SIZE(wasmtime_fiber_init)
TYPE(wasmtime_fiber_start)
FUNCTION(wasmtime_fiber_start):
.cfi_startproc simple
.cfi_escape 0x0f, /* DW_CFA_def_cfa_expression */ \
5, /* the byte length of this expression */ \
0x74, 0x08, /* DW_OP_breg4 (%esp) + 8 */ \
0x06, /* DW_OP_deref */ \
0x23, 0x14 /* DW_OP_plus_uconst 0x14 */
.cfi_rel_offset eip, -4
.cfi_rel_offset ebp, -8
.cfi_rel_offset ebx, -12
.cfi_rel_offset esi, -16
.cfi_rel_offset edi, -20
// Our arguments and stack alignment are all prepped by
// `wasmtime_fiber_init`.
call *%ebp
ud2
.cfi_endproc
SIZE(wasmtime_fiber_start)
FOOTER

159
crates/fiber/src/arch/x86_64.S Normal file
View File

@@ -0,0 +1,159 @@
// A WORD OF CAUTION
//
// This entire file basically needs to be kept in sync with itself. It's not
// really possible to modify just one bit of this file without understanding
// all the other bits. Documentation tries to reference various bits here and
// there but try to make sure to read over everything before tweaking things!
#include "header.h"
// fn(top_of_stack(%rdi): *mut u8)
HIDDEN(wasmtime_fiber_switch)
GLOBL(wasmtime_fiber_switch)
.align 16
TYPE(wasmtime_fiber_switch)
FUNCTION(wasmtime_fiber_switch):
// We're switching to arbitrary code somewhere else, so pessimistically
// assume that all callee-save register are clobbered. This means we need
// to save/restore all of them.
//
// Note that this order for saving is important since we use CFI directives
// below to point to where all the saved registers are.
pushq %rbp
pushq %rbx
pushq %r12
pushq %r13
pushq %r14
pushq %r15
// Load pointer that we're going to resume at and store where we're going
// to get resumed from. This is in accordance with the diagram at the top
// of unix.rs.
movq -0x10(%rdi), %rax
mov %rsp, -0x10(%rdi)
// Swap stacks and restore all our callee-saved registers
mov %rax, %rsp
popq %r15
popq %r14
popq %r13
popq %r12
popq %rbx
popq %rbp
ret
SIZE(wasmtime_fiber_switch)
// fn(
// top_of_stack(%rdi): *mut u8,
// entry_point(%rsi): extern fn(*mut u8, *mut u8),
// entry_arg0(%rdx): *mut u8,
// )
HIDDEN(wasmtime_fiber_init)
GLOBL(wasmtime_fiber_init)
.align 16
TYPE(wasmtime_fiber_init)
FUNCTION(wasmtime_fiber_init):
// Here we're going to set up a stack frame as expected by
// `wasmtime_fiber_switch`. The values we store here will get restored into
// registers by that function and the `wasmtime_fiber_start` function will
// take over and understands which values are in which registers.
//
// The first 16 bytes of stack are reserved for metadata, so we start
// storing values beneath that.
lea FUNCTION(wasmtime_fiber_start)(%rip), %rax
movq %rax, -0x18(%rdi)
movq %rdi, -0x20(%rdi) // loaded into rbp during switch
movq %rsi, -0x28(%rdi) // loaded into rbx during switch
movq %rdx, -0x30(%rdi) // loaded into r12 during switch
// And then we specify the stack pointer resumption should begin at. Our
// `wasmtime_fiber_switch` function consumes 6 registers plus a return
// pointer, and the top 16 bytes aree resereved, so that's:
//
// (6 + 1) * 16 + 16 = 0x48
lea -0x48(%rdi), %rax
movq %rax, -0x10(%rdi)
ret
SIZE(wasmtime_fiber_init)
// This is a pretty special function that has no real signature. Its use is to
// be the "base" function of all fibers. This entrypoint is used in
// `wasmtime_fiber_init` to bootstrap the execution of a new fiber.
//
// We also use this function as a persistent frame on the stack to emit dwarf
// information to unwind into the caller. This allows us to unwind from the
// fiber's stack back to the main stack that the fiber was called from. We use
// special dwarf directives here to do so since this is a pretty nonstandard
// function.
//
// If you're curious a decent introduction to CFI things and unwinding is at
// https://www.imperialviolet.org/2017/01/18/cfi.html
.align 16
TYPE(wasmtime_fiber_start)
FUNCTION(wasmtime_fiber_start):
// Use the `simple` directive on the startproc here which indicates that some
// default settings for the platform are omitted, since this function is so
// nonstandard
.cfi_startproc simple
// This is where things get special, we're specifying a custom dwarf
// expression for how to calculate the CFA. The goal here is that we need
// to load the parent's stack pointer just before the call it made into
// `wasmtime_fiber_switch`. Note that the CFA value changes over time as
// well because a fiber may be resumed multiple times from different points
// on the original stack. This means that our custom CFA directive involves
// `DW_OP_deref`, which loads data from memory.
//
// The expression we're encoding here is that the CFA, the stack pointer of
// whatever called into `wasmtime_fiber_start`, is:
//
// *$rsp + 0x38
//
// $rsp is the stack pointer of `wasmtime_fiber_start` at the time the next
// instruction after the `.cfi_escape` is executed. Our $rsp at the start
// of this function is 16 bytes below the top of the stack (0xAff0 in
// the diagram in unix.rs). The $rsp to resume at is stored at that
// location, so we dereference the stack pointer to load it.
//
// After dereferencing, though, we have the $rsp value for
// `wasmtime_fiber_switch` itself. That's a weird function which sort of
// and sort of doesn't exist on the stack. We want to point to the caller
// of `wasmtime_fiber_switch`, so to do that we need to skip the stack space
// reserved by `wasmtime_fiber_switch`, which is the 6 saved registers plus
// the return address of the caller's `call` instruction. Hence we offset
// another 0x38 bytes.
.cfi_escape 0x0f, /* DW_CFA_def_cfa_expression */ \
4, /* the byte length of this expression */ \
0x57, /* DW_OP_reg7 (%rsp) */ \
0x06, /* DW_OP_deref */ \
0x23, 0x38 /* DW_OP_plus_uconst 0x38 */
// And now after we've indicated where our CFA is for our parent function,
// we can define that where all of the saved registers are located. This
// uses standard `.cfi` directives which indicate that these registers are
// all stored relative to the CFA. Note that this order is kept in sync
// with the above register spills in `wasmtime_fiber_switch`.
.cfi_rel_offset rip, -8
.cfi_rel_offset rbp, -16
.cfi_rel_offset rbx, -24
.cfi_rel_offset r12, -32
.cfi_rel_offset r13, -40
.cfi_rel_offset r14, -48
.cfi_rel_offset r15, -56
// The body of this function is pretty similar. All our parameters are
// already loaded into registers by the switch function. The
// `wasmtime_fiber_init` routine arranged the various values to be
// materialized into the registers used here. Our job is to then move the
// values into the ABI-defined registers and call the entry-point. Note that
// `callq` is used here to leave this frame on the stack so we can use the
// dwarf info here for unwinding. The trailing `ud2` is just for safety.
mov %r12,%rdi
mov %rbp,%rsi
callq *%rbx
ud2
.cfi_endproc
SIZE(wasmtime_fiber_start)
FOOTER

249
crates/fiber/src/lib.rs Normal file
View File

@@ -0,0 +1,249 @@
use std::any::Any;
use std::cell::Cell;
use std::io;
use std::marker::PhantomData;
use std::panic::{self, AssertUnwindSafe};
#[cfg(windows)]
mod windows;
#[cfg(windows)]
use windows as imp;
#[cfg(unix)]
mod unix;
#[cfg(unix)]
use unix as imp;
pub struct Fiber<'a, Resume, Yield, Return> {
inner: imp::Fiber,
done: Cell<bool>,
_phantom: PhantomData<&'a (Resume, Yield, Return)>,
}
pub struct Suspend<Resume, Yield, Return> {
inner: imp::Suspend,
_phantom: PhantomData<(Resume, Yield, Return)>,
}
enum RunResult<Resume, Yield, Return> {
Executing,
Resuming(Resume),
Yield(Yield),
Returned(Return),
Panicked(Box<dyn Any + Send>),
}
impl<'a, Resume, Yield, Return> Fiber<'a, Resume, Yield, Return> {
/// Creates a new fiber which will execute `func` on a new native stack of
/// size `stack_size`.
///
/// This function returns a `Fiber` which, when resumed, will execute `func`
/// to completion. When desired the `func` can suspend itself via
/// `Fiber::suspend`.
pub fn new(
stack_size: usize,
func: impl FnOnce(Resume, &Suspend<Resume, Yield, Return>) -> Return + 'a,
) -> io::Result<Fiber<'a, Resume, Yield, Return>> {
Ok(Fiber {
inner: imp::Fiber::new(stack_size, func)?,
done: Cell::new(false),
_phantom: PhantomData,
})
}
/// Resumes execution of this fiber.
///
/// This function will transfer execution to the fiber and resume from where
/// it last left off.
///
/// Returns `true` if the fiber finished or `false` if the fiber was
/// suspended in the middle of execution.
///
/// # Panics
///
/// Panics if the current thread is already executing a fiber or if this
/// fiber has already finished.
///
/// Note that if the fiber itself panics during execution then the panic
/// will be propagated to this caller.
pub fn resume(&self, val: Resume) -> Result<Return, Yield> {
assert!(!self.done.replace(true), "cannot resume a finished fiber");
let result = Cell::new(RunResult::Resuming(val));
self.inner.resume(&result);
match result.into_inner() {
RunResult::Resuming(_) | RunResult::Executing => unreachable!(),
RunResult::Yield(y) => {
self.done.set(false);
Err(y)
}
RunResult::Returned(r) => Ok(r),
RunResult::Panicked(payload) => std::panic::resume_unwind(payload),
}
}
/// Returns whether this fiber has finished executing.
pub fn done(&self) -> bool {
self.done.get()
}
}
impl<Resume, Yield, Return> Suspend<Resume, Yield, Return> {
/// Suspend execution of a currently running fiber.
///
/// This function will switch control back to the original caller of
/// `Fiber::resume`. This function will then return once the `Fiber::resume`
/// function is called again.
///
/// # Panics
///
/// Panics if the current thread is not executing a fiber from this library.
pub fn suspend(&self, value: Yield) -> Resume {
self.inner
.switch::<Resume, Yield, Return>(RunResult::Yield(value))
}
fn execute(
inner: imp::Suspend,
initial: Resume,
func: impl FnOnce(Resume, &Suspend<Resume, Yield, Return>) -> Return,
) {
let suspend = Suspend {
inner,
_phantom: PhantomData,
};
let result = panic::catch_unwind(AssertUnwindSafe(|| (func)(initial, &suspend)));
suspend.inner.switch::<Resume, Yield, Return>(match result {
Ok(result) => RunResult::Returned(result),
Err(panic) => RunResult::Panicked(panic),
});
}
}
impl<A, B, C> Drop for Fiber<'_, A, B, C> {
fn drop(&mut self) {
debug_assert!(self.done.get(), "fiber dropped without finishing");
}
}
#[cfg(test)]
mod tests {
use super::Fiber;
use std::cell::Cell;
use std::panic::{self, AssertUnwindSafe};
use std::rc::Rc;
#[test]
fn small_stacks() {
Fiber::<(), (), ()>::new(0, |_, _| {})
.unwrap()
.resume(())
.unwrap();
Fiber::<(), (), ()>::new(1, |_, _| {})
.unwrap()
.resume(())
.unwrap();
}
#[test]
fn smoke() {
let hit = Rc::new(Cell::new(false));
let hit2 = hit.clone();
let fiber = Fiber::<(), (), ()>::new(1024 * 1024, move |_, _| {
hit2.set(true);
})
.unwrap();
assert!(!hit.get());
fiber.resume(()).unwrap();
assert!(hit.get());
}
#[test]
fn suspend_and_resume() {
let hit = Rc::new(Cell::new(false));
let hit2 = hit.clone();
let fiber = Fiber::<(), (), ()>::new(1024 * 1024, move |_, s| {
s.suspend(());
hit2.set(true);
s.suspend(());
})
.unwrap();
assert!(!hit.get());
assert!(fiber.resume(()).is_err());
assert!(!hit.get());
assert!(fiber.resume(()).is_err());
assert!(hit.get());
assert!(fiber.resume(()).is_ok());
assert!(hit.get());
}
#[test]
fn backtrace_traces_to_host() {
#[inline(never)] // try to get this to show up in backtraces
fn look_for_me() {
run_test();
}
fn assert_contains_host() {
let trace = backtrace::Backtrace::new();
println!("{:?}", trace);
assert!(
trace
.frames()
.iter()
.flat_map(|f| f.symbols())
.filter_map(|s| Some(s.name()?.to_string()))
.any(|s| s.contains("look_for_me"))
// TODO: apparently windows unwind routines don't unwind through fibers, so this will always fail. Is there a way we can fix that?
|| cfg!(windows)
);
}
fn run_test() {
let fiber = Fiber::<(), (), ()>::new(1024 * 1024, move |(), s| {
assert_contains_host();
s.suspend(());
assert_contains_host();
s.suspend(());
assert_contains_host();
})
.unwrap();
assert!(fiber.resume(()).is_err());
assert!(fiber.resume(()).is_err());
assert!(fiber.resume(()).is_ok());
}
look_for_me();
}
#[test]
fn panics_propagated() {
let a = Rc::new(Cell::new(false));
let b = SetOnDrop(a.clone());
let fiber = Fiber::<(), (), ()>::new(1024 * 1024, move |(), _s| {
drop(&b);
panic!();
})
.unwrap();
assert!(panic::catch_unwind(AssertUnwindSafe(|| fiber.resume(()))).is_err());
assert!(a.get());
struct SetOnDrop(Rc<Cell<bool>>);
impl Drop for SetOnDrop {
fn drop(&mut self) {
self.0.set(true);
}
}
}
#[test]
fn suspend_and_resume_values() {
let fiber = Fiber::new(1024 * 1024, move |first, s| {
assert_eq!(first, 2.0);
assert_eq!(s.suspend(4), 3.0);
"hello".to_string()
})
.unwrap();
assert_eq!(fiber.resume(2.0), Err(4));
assert_eq!(fiber.resume(3.0), Ok("hello".to_string()));
}
}

174
crates/fiber/src/unix.rs Normal file
View File

@@ -0,0 +1,174 @@
//! The unix fiber implementation has some platform-specific details
//! (naturally) but there's a few details of the stack layout which are common
//! amongst all platforms using this file. Remember that none of this applies to
//! Windows, which is entirely separate.
//!
//! The stack is expected to look pretty standard with a guard page at the end.
//! Currently allocation happens in this file but this is probably going to be
//! refactored to happen somewhere else. Otherwise though the stack layout is
//! expected to look like so:
//!
//!
//! ```text
//! 0xB000 +-----------------------+ <- top of stack
//! | &Cell<RunResult> | <- where to store results
//! 0xAff8 +-----------------------+
//! | *const u8 | <- last sp to resume from
//! 0xAff0 +-----------------------+ <- 16-byte aligned
//! | |
//! ~ ... ~ <- actual native stack space to use
//! | |
//! 0x1000 +-----------------------+
//! | guard page |
//! 0x0000 +-----------------------+
//! ```
//!
//! Here `0xAff8` is filled in temporarily while `resume` is running. The fiber
//! started with 0xB000 as a parameter so it knows how to find this.
//! Additionally `resumes` stores state at 0xAff0 to restart execution, and
//! `suspend`, which has 0xB000 so it can find this, will read that and write
//! its own resumption information into this slot as well.
use crate::RunResult;
use std::cell::Cell;
use std::io;
use std::ptr;
pub struct Fiber {
// Description of the mmap region we own. This should be abstracted
// eventually so we aren't personally mmap-ing this region.
mmap: *mut libc::c_void,
mmap_len: usize,
}
pub struct Suspend {
top_of_stack: *mut u8,
}
extern "C" {
fn wasmtime_fiber_init(
top_of_stack: *mut u8,
entry: extern "C" fn(*mut u8, *mut u8),
entry_arg0: *mut u8,
);
fn wasmtime_fiber_switch(top_of_stack: *mut u8);
}
extern "C" fn fiber_start<F, A, B, C>(arg0: *mut u8, top_of_stack: *mut u8)
where
F: FnOnce(A, &super::Suspend<A, B, C>) -> C,
{
unsafe {
let inner = Suspend { top_of_stack };
let initial = inner.take_resume::<A, B, C>();
super::Suspend::<A, B, C>::execute(inner, initial, Box::from_raw(arg0.cast::<F>()))
}
}
impl Fiber {
pub fn new<F, A, B, C>(stack_size: usize, func: F) -> io::Result<Fiber>
where
F: FnOnce(A, &super::Suspend<A, B, C>) -> C,
{
let fiber = Fiber::alloc_with_stack(stack_size)?;
unsafe {
// Initialize the top of the stack to be resumed from
let top_of_stack = fiber.top_of_stack();
let data = Box::into_raw(Box::new(func)).cast();
wasmtime_fiber_init(top_of_stack, fiber_start::<F, A, B, C>, data);
Ok(fiber)
}
}
fn alloc_with_stack(stack_size: usize) -> io::Result<Fiber> {
unsafe {
// Round up our stack size request to the nearest multiple of the
// page size.
let page_size = libc::sysconf(libc::_SC_PAGESIZE) as usize;
let stack_size = if stack_size == 0 {
page_size
} else {
(stack_size + (page_size - 1)) & (!(page_size - 1))
};
// Add in one page for a guard page and then ask for some memory.
let mmap_len = stack_size + page_size;
let mmap = libc::mmap(
ptr::null_mut(),
mmap_len,
libc::PROT_NONE,
libc::MAP_ANON | libc::MAP_PRIVATE,
-1,
0,
);
if mmap == libc::MAP_FAILED {
return Err(io::Error::last_os_error());
}
let ret = Fiber { mmap, mmap_len };
let res = libc::mprotect(
mmap.cast::<u8>().add(page_size).cast(),
stack_size,
libc::PROT_READ | libc::PROT_WRITE,
);
if res != 0 {
Err(io::Error::last_os_error())
} else {
Ok(ret)
}
}
}
pub(crate) fn resume<A, B, C>(&self, result: &Cell<RunResult<A, B, C>>) {
unsafe {
// Store where our result is going at the very tip-top of the
// stack, otherwise known as our reserved slot for this information.
//
// In the diagram above this is updating address 0xAff8
let top_of_stack = self.top_of_stack();
let addr = top_of_stack.cast::<usize>().offset(-1);
addr.write(result as *const _ as usize);
wasmtime_fiber_switch(top_of_stack);
// null this out to help catch use-after-free
addr.write(0);
}
}
unsafe fn top_of_stack(&self) -> *mut u8 {
self.mmap.cast::<u8>().add(self.mmap_len)
}
}
impl Drop for Fiber {
fn drop(&mut self) {
unsafe {
let ret = libc::munmap(self.mmap, self.mmap_len);
debug_assert!(ret == 0);
}
}
}
impl Suspend {
pub(crate) fn switch<A, B, C>(&self, result: RunResult<A, B, C>) -> A {
unsafe {
// Calculate 0xAff8 and then write to it
(*self.result_location::<A, B, C>()).set(result);
wasmtime_fiber_switch(self.top_of_stack);
self.take_resume::<A, B, C>()
}
}
unsafe fn take_resume<A, B, C>(&self) -> A {
match (*self.result_location::<A, B, C>()).replace(RunResult::Executing) {
RunResult::Resuming(val) => val,
_ => panic!("not in resuming state"),
}
}
unsafe fn result_location<A, B, C>(&self) -> *const Cell<RunResult<A, B, C>> {
let ret = self.top_of_stack.cast::<*const u8>().offset(-1).read();
assert!(!ret.is_null());
return ret.cast();
}
}

130
crates/fiber/src/windows.rs Normal file
View File

@@ -0,0 +1,130 @@
use crate::RunResult;
use std::cell::Cell;
use std::io;
use std::ptr;
use winapi::shared::minwindef::*;
use winapi::um::fibersapi::*;
use winapi::um::winbase::*;
pub struct Fiber {
fiber: LPVOID,
state: Box<StartState>,
}
pub struct Suspend {
state: *const StartState,
}
struct StartState {
parent: Cell<LPVOID>,
initial_closure: Cell<*mut u8>,
result_location: Cell<*const u8>,
}
const FIBER_FLAG_FLOAT_SWITCH: DWORD = 1;
extern "C" {
fn wasmtime_fiber_get_current() -> LPVOID;
}
unsafe extern "system" fn fiber_start<F, A, B, C>(data: LPVOID)
where
F: FnOnce(A, &super::Suspend<A, B, C>) -> C,
{
let state = data.cast::<StartState>();
let func = Box::from_raw((*state).initial_closure.get().cast::<F>());
(*state).initial_closure.set(ptr::null_mut());
let suspend = Suspend { state };
let initial = suspend.take_resume::<A, B, C>();
super::Suspend::<A, B, C>::execute(suspend, initial, *func);
}
impl Fiber {
pub fn new<F, A, B, C>(stack_size: usize, func: F) -> io::Result<Fiber>
where
F: FnOnce(A, &super::Suspend<A, B, C>) -> C,
{
unsafe {
let state = Box::new(StartState {
initial_closure: Cell::new(Box::into_raw(Box::new(func)).cast()),
parent: Cell::new(ptr::null_mut()),
result_location: Cell::new(ptr::null()),
});
let fiber = CreateFiberEx(
0,
stack_size,
FIBER_FLAG_FLOAT_SWITCH,
Some(fiber_start::<F, A, B, C>),
&*state as *const StartState as *mut _,
);
if fiber.is_null() {
drop(Box::from_raw(state.initial_closure.get().cast::<F>()));
Err(io::Error::last_os_error())
} else {
Ok(Fiber { fiber, state })
}
}
}
pub(crate) fn resume<A, B, C>(&self, result: &Cell<RunResult<A, B, C>>) {
unsafe {
let is_fiber = IsThreadAFiber() != 0;
let parent_fiber = if is_fiber {
wasmtime_fiber_get_current()
} else {
ConvertThreadToFiber(ptr::null_mut())
};
assert!(
!parent_fiber.is_null(),
"failed to make current thread a fiber"
);
self.state
.result_location
.set(result as *const _ as *const _);
self.state.parent.set(parent_fiber);
SwitchToFiber(self.fiber);
self.state.parent.set(ptr::null_mut());
self.state.result_location.set(ptr::null());
if !is_fiber {
let res = ConvertFiberToThread();
assert!(res != 0, "failed to convert main thread back");
}
}
}
}
impl Drop for Fiber {
fn drop(&mut self) {
unsafe {
DeleteFiber(self.fiber);
}
}
}
impl Suspend {
pub(crate) fn switch<A, B, C>(&self, result: RunResult<A, B, C>) -> A {
unsafe {
(*self.result_location::<A, B, C>()).set(result);
debug_assert!(IsThreadAFiber() != 0);
let parent = (*self.state).parent.get();
debug_assert!(!parent.is_null());
SwitchToFiber(parent);
self.take_resume::<A, B, C>()
}
}
unsafe fn take_resume<A, B, C>(&self) -> A {
match (*self.result_location::<A, B, C>()).replace(RunResult::Executing) {
RunResult::Resuming(val) => val,
_ => panic!("not in resuming state"),
}
}
unsafe fn result_location<A, B, C>(&self) -> *const Cell<RunResult<A, B, C>> {
let ret = (*self.state)
.result_location
.get()
.cast::<Cell<RunResult<A, B, C>>>();
assert!(!ret.is_null());
return ret;
}
}