Return anyhow::Error from host functions instead of Trap, redesign Trap (#5149)

* Return `anyhow::Error` from host functions instead of `Trap`

This commit refactors how errors are modeled when returned from host
functions and additionally refactors how custom errors work with `Trap`.
At a high level functions in Wasmtime that previously worked with
`Result<T, Trap>` now work with `Result<T>` instead where the error is
`anyhow::Error`. This includes functions such as:

* Host-defined functions in a `Linker<T>`
* `TypedFunc::call`
* Host-related callbacks like call hooks

Errors are now modeled primarily as `anyhow::Error` throughout Wasmtime.
This subsequently removes the need for `Trap` to have the ability to
represent all host-defined errors as it previously did. Consequently the
`From` implementations for any error into a `Trap` have been removed
here and the only embedder-defined way to create a `Trap` is to use
`Trap::new` with a custom string.

After this commit the distinction between a `Trap` and a host error is
the wasm backtrace that it contains. Previously all errors in host
functions would flow through a `Trap` and get a wasm backtrace attached
to them, but now this only happens if a `Trap` itself is created meaning
that arbitrary host-defined errors flowing from a host import to the
other side won't get backtraces attached. Some internals of Wasmtime
itself were updated or preserved to use `Trap::new` to capture a
backtrace where it seemed useful, such as when fuel runs out.

The main motivation for this commit is that it now enables hosts to
thread a concrete error type from a host function all the way through to
where a wasm function was invoked. Previously this could not be done
since the host error was wrapped in a `Trap` that didn't provide the
ability to get at the internals.

A consequence of this commit is that when a host error is returned that
isn't a `Trap` we'll capture a backtrace and then won't have a `Trap` to
attach it to. To avoid losing the contextual information this commit
uses the `Error::context` method to attach the backtrace as contextual
information to ensure that the backtrace is itself not lost.

This is a breaking change for likely all users of Wasmtime, but it's
hoped to be a relatively minor change to workaround. Most use cases can
likely change `-> Result<T, Trap>` to `-> Result<T>` and otherwise
explicit creation of a `Trap` is largely no longer necessary.

* Fix some doc links

* add some tests and make a backtrace type public (#55)

* Trap: avoid a trailing newline in the Display impl

which in turn ends up with three newlines between the end of the
backtrace and the `Caused by` in the anyhow Debug impl

* make BacktraceContext pub, and add tests showing downcasting behavior of anyhow::Error to traps or backtraces

* Remove now-unnecesary `Trap` downcasts in `Linker::module`

* Fix test output expectations

* Remove `Trap::i32_exit`

This commit removes special-handling in the `wasmtime::Trap` type for
the i32 exit code required by WASI. This is now instead modeled as a
specific `I32Exit` error type in the `wasmtime-wasi` crate which is
returned by the `proc_exit` hostcall. Embedders which previously tested
for i32 exits now downcast to the `I32Exit` value.

* Remove the `Trap::new` constructor

This commit removes the ability to create a trap with an arbitrary error
message. The purpose of this commit is to continue the prior trend of
leaning into the `anyhow::Error` type instead of trying to recreate it
with `Trap`. A subsequent simplification to `Trap` after this commit is
that `Trap` will simply be an `enum` of trap codes with no extra
information. This commit is doubly-motivated by the desire to always use
the new `BacktraceContext` type instead of sometimes using that and
sometimes using `Trap`.

Most of the changes here were around updating `Trap::new` calls to
`bail!` calls instead. Tests which assert particular error messages
additionally often needed to use the `:?` formatter instead of the `{}`
formatter because the prior formats the whole `anyhow::Error` and the
latter only formats the top-most error, which now contains the
backtrace.

* Merge `Trap` and `TrapCode`

With prior refactorings there's no more need for `Trap` to be opaque or
otherwise contain a backtrace. This commit parse down `Trap` to simply
an `enum` which was the old `TrapCode`. All various tests and such were
updated to handle this.

The main consequence of this commit is that all errors have a
`BacktraceContext` context attached to them. This unfortunately means
that the backtrace is printed first before the error message or trap
code, but given all the prior simplifications that seems worth it at
this time.

* Rename `BacktraceContext` to `WasmBacktrace`

This feels like a better name given how this has turned out, and
additionally this commit removes having both `WasmBacktrace` and
`BacktraceContext`.

* Soup up documentation for errors and traps

* Fix build of the C API

Co-authored-by: Pat Hickey <pat@moreproductive.org>

crates/c-api/src/func.rs

@@ -3,6 +3,8 @@ use crate::{
     wasm_extern_t, wasm_functype_t, wasm_store_t, wasm_val_t, wasm_val_vec_t, wasmtime_error_t,
     wasmtime_extern_t, wasmtime_val_t, wasmtime_val_union, CStoreContext, CStoreContextMut,
 };
+use anyhow::{Error, Result};
+use std::any::Any;
 use std::ffi::c_void;
 use std::mem::{self, MaybeUninit};
 use std::panic::{self, AssertUnwindSafe};
@@ -67,7 +69,7 @@ unsafe fn create_function(
             let mut out_results: wasm_val_vec_t = vec![wasm_val_t::default(); results.len()].into();
             let out = func(&params, &mut out_results);
             if let Some(trap) = out {
-                return Err(trap.trap.clone());
+                return Err(trap.error);
             }
 
             let out_results = out_results.as_slice();
@@ -152,24 +154,25 @@ pub unsafe extern "C" fn wasm_func_call(
             }
             ptr::null_mut()
         }
-        Ok(Err(trap)) => match trap.downcast::<Trap>() {
-            Ok(trap) => Box::into_raw(Box::new(wasm_trap_t::new(trap))),
-            Err(err) => Box::into_raw(Box::new(wasm_trap_t::new(err.into()))),
-        },
+        Ok(Err(err)) => Box::into_raw(Box::new(wasm_trap_t::new(err))),
         Err(panic) => {
-            let trap = if let Some(msg) = panic.downcast_ref::<String>() {
-                Trap::new(msg)
-            } else if let Some(msg) = panic.downcast_ref::<&'static str>() {
-                Trap::new(*msg)
-            } else {
-                Trap::new("rust panic happened")
-            };
-            let trap = Box::new(wasm_trap_t::new(trap));
+            let err = error_from_panic(panic);
+            let trap = Box::new(wasm_trap_t::new(err));
             Box::into_raw(trap)
         }
     }
 }
 
+fn error_from_panic(panic: Box<dyn Any + Send>) -> Error {
+    if let Some(msg) = panic.downcast_ref::<String>() {
+        Error::msg(msg.clone())
+    } else if let Some(msg) = panic.downcast_ref::<&'static str>() {
+        Error::msg(*msg)
+    } else {
+        Error::msg("rust panic happened")
+    }
+}
+
 #[no_mangle]
 pub unsafe extern "C" fn wasm_func_type(f: &wasm_func_t) -> Box<wasm_functype_t> {
     Box::new(wasm_functype_t::new(f.func().ty(f.ext.store.context())))
@@ -235,7 +238,7 @@ pub(crate) unsafe fn c_callback_to_rust_fn(
     callback: wasmtime_func_callback_t,
     data: *mut c_void,
     finalizer: Option<extern "C" fn(*mut std::ffi::c_void)>,
-) -> impl Fn(Caller<'_, crate::StoreData>, &[Val], &mut [Val]) -> Result<(), Trap> {
+) -> impl Fn(Caller<'_, crate::StoreData>, &[Val], &mut [Val]) -> Result<()> {
     let foreign = crate::ForeignData { data, finalizer };
     move |mut caller, params, results| {
         drop(&foreign); // move entire foreign into this closure
@@ -264,7 +267,7 @@ pub(crate) unsafe fn c_callback_to_rust_fn(
             out_results.len(),
         );
         if let Some(trap) = out {
-            return Err(trap.trap);
+            return Err(trap.error);
         }
 
         // Translate the `wasmtime_val_t` results into the `results` space
@@ -299,14 +302,14 @@ pub(crate) unsafe fn c_unchecked_callback_to_rust_fn(
     callback: wasmtime_func_unchecked_callback_t,
     data: *mut c_void,
     finalizer: Option<extern "C" fn(*mut std::ffi::c_void)>,
-) -> impl Fn(Caller<'_, crate::StoreData>, &mut [ValRaw]) -> Result<(), Trap> {
+) -> impl Fn(Caller<'_, crate::StoreData>, &mut [ValRaw]) -> Result<()> {
     let foreign = crate::ForeignData { data, finalizer };
     move |caller, values| {
         drop(&foreign); // move entire foreign into this closure
         let mut caller = wasmtime_caller_t { caller };
         match callback(foreign.data, &mut caller, values.as_mut_ptr(), values.len()) {
             None => Ok(()),
-            Some(trap) => Err(trap.trap),
+            Some(trap) => Err(trap.error),
         }
     }
 }
@@ -348,22 +351,17 @@ pub unsafe extern "C" fn wasmtime_func_call(
             store.data_mut().wasm_val_storage = params;
             None
         }
-        Ok(Err(trap)) => match trap.downcast::<Trap>() {
-            Ok(trap) => {
+        Ok(Err(trap)) => {
+            if trap.is::<Trap>() {
                 *trap_ret = Box::into_raw(Box::new(wasm_trap_t::new(trap)));
                 None
-            }
-            Err(err) => Some(Box::new(wasmtime_error_t::from(err))),
-        },
-        Err(panic) => {
-            let trap = if let Some(msg) = panic.downcast_ref::<String>() {
-                Trap::new(msg)
-            } else if let Some(msg) = panic.downcast_ref::<&'static str>() {
-                Trap::new(*msg)
             } else {
-                Trap::new("rust panic happened")
-            };
-            *trap_ret = Box::into_raw(Box::new(wasm_trap_t::new(trap)));
+                Some(Box::new(wasmtime_error_t::from(trap)))
+            }
+        }
+        Err(panic) => {
+            let err = error_from_panic(panic);
+            *trap_ret = Box::into_raw(Box::new(wasm_trap_t::new(err)));
             None
         }
     }

crates/c-api/src/instance.rs

@@ -41,7 +41,7 @@ pub unsafe extern "C" fn wasm_instance_new(
         ))),
         Err(e) => {
             if let Some(ptr) = result {
-                *ptr = Box::into_raw(Box::new(wasm_trap_t::new(e.into())));
+                *ptr = Box::into_raw(Box::new(wasm_trap_t::new(e)));
             }
             None
         }
@@ -100,7 +100,7 @@ pub(crate) fn handle_instantiate(
         }
         Err(e) => match e.downcast::<Trap>() {
             Ok(trap) => {
-                *trap_ptr = Box::into_raw(Box::new(wasm_trap_t::new(trap)));
+                *trap_ptr = Box::into_raw(Box::new(wasm_trap_t::new(trap.into())));
                 None
             }
             Err(e) => Some(Box::new(e.into())),

crates/c-api/src/trap.rs

@@ -1,25 +1,37 @@
 use crate::{wasm_frame_vec_t, wasm_instance_t, wasm_name_t, wasm_store_t};
+use anyhow::{anyhow, Error};
 use once_cell::unsync::OnceCell;
-use wasmtime::{Trap, TrapCode};
+use wasmtime::{Trap, WasmBacktrace};
 
 #[repr(C)]
-#[derive(Clone)]
 pub struct wasm_trap_t {
-    pub(crate) trap: Trap,
+    pub(crate) error: Error,
+}
+
+// This is currently only needed for the `wasm_trap_copy` API in the C API.
+//
+// For now the impl here is "fake it til you make it" since this is losing
+// context by only cloning the error string.
+impl Clone for wasm_trap_t {
+    fn clone(&self) -> wasm_trap_t {
+        wasm_trap_t {
+            error: anyhow!("{:?}", self.error),
+        }
+    }
 }
 
 wasmtime_c_api_macros::declare_ref!(wasm_trap_t);
 
 impl wasm_trap_t {
-    pub(crate) fn new(trap: Trap) -> wasm_trap_t {
-        wasm_trap_t { trap: trap }
+    pub(crate) fn new(error: Error) -> wasm_trap_t {
+        wasm_trap_t { error }
     }
 }
 
 #[repr(C)]
 #[derive(Clone)]
-pub struct wasm_frame_t {
-    trap: Trap,
+pub struct wasm_frame_t<'a> {
+    trace: &'a WasmBacktrace,
     idx: usize,
     func_name: OnceCell<Option<wasm_name_t>>,
     module_name: OnceCell<Option<wasm_name_t>>,
@@ -40,7 +52,7 @@ pub extern "C" fn wasm_trap_new(
     }
     let message = String::from_utf8_lossy(&message[..message.len() - 1]);
     Box::new(wasm_trap_t {
-        trap: Trap::new(message),
+        error: Error::msg(message.into_owned()),
     })
 }
 
@@ -49,24 +61,28 @@ pub unsafe extern "C" fn wasmtime_trap_new(message: *const u8, len: usize) -> Bo
     let bytes = crate::slice_from_raw_parts(message, len);
     let message = String::from_utf8_lossy(&bytes);
     Box::new(wasm_trap_t {
-        trap: Trap::new(message),
+        error: Error::msg(message.into_owned()),
     })
 }
 
 #[no_mangle]
 pub extern "C" fn wasm_trap_message(trap: &wasm_trap_t, out: &mut wasm_message_t) {
     let mut buffer = Vec::new();
-    buffer.extend_from_slice(trap.trap.to_string().as_bytes());
+    buffer.extend_from_slice(format!("{:?}", trap.error).as_bytes());
     buffer.reserve_exact(1);
     buffer.push(0);
     out.set_buffer(buffer);
 }
 
 #[no_mangle]
-pub extern "C" fn wasm_trap_origin(raw: &wasm_trap_t) -> Option<Box<wasm_frame_t>> {
-    if raw.trap.trace().unwrap_or(&[]).len() > 0 {
+pub extern "C" fn wasm_trap_origin(raw: &wasm_trap_t) -> Option<Box<wasm_frame_t<'_>>> {
+    let trace = match raw.error.downcast_ref::<WasmBacktrace>() {
+        Some(trap) => trap,
+        None => return None,
+    };
+    if trace.frames().len() > 0 {
         Some(Box::new(wasm_frame_t {
-            trap: raw.trap.clone(),
+            trace,
             idx: 0,
             func_name: OnceCell::new(),
             module_name: OnceCell::new(),
@@ -77,11 +93,15 @@ pub extern "C" fn wasm_trap_origin(raw: &wasm_trap_t) -> Option<Box<wasm_frame_t
 }
 
 #[no_mangle]
-pub extern "C" fn wasm_trap_trace(raw: &wasm_trap_t, out: &mut wasm_frame_vec_t) {
-    let vec = (0..raw.trap.trace().unwrap_or(&[]).len())
+pub extern "C" fn wasm_trap_trace<'a>(raw: &'a wasm_trap_t, out: &mut wasm_frame_vec_t<'a>) {
+    let trace = match raw.error.downcast_ref::<WasmBacktrace>() {
+        Some(trap) => trap,
+        None => return out.set_buffer(Vec::new()),
+    };
+    let vec = (0..trace.frames().len())
         .map(|idx| {
             Some(Box::new(wasm_frame_t {
-                trap: raw.trap.clone(),
+                trace,
                 idx,
                 func_name: OnceCell::new(),
                 module_name: OnceCell::new(),
@@ -93,50 +113,54 @@ pub extern "C" fn wasm_trap_trace(raw: &wasm_trap_t, out: &mut wasm_frame_vec_t)
 
 #[no_mangle]
 pub extern "C" fn wasmtime_trap_code(raw: &wasm_trap_t, code: &mut i32) -> bool {
-    match raw.trap.trap_code() {
-        Some(c) => {
-            *code = match c {
-                TrapCode::StackOverflow => 0,
-                TrapCode::MemoryOutOfBounds => 1,
-                TrapCode::HeapMisaligned => 2,
-                TrapCode::TableOutOfBounds => 3,
-                TrapCode::IndirectCallToNull => 4,
-                TrapCode::BadSignature => 5,
-                TrapCode::IntegerOverflow => 6,
-                TrapCode::IntegerDivisionByZero => 7,
-                TrapCode::BadConversionToInteger => 8,
-                TrapCode::UnreachableCodeReached => 9,
-                TrapCode::Interrupt => 10,
-                _ => unreachable!(),
-            };
-            true
-        }
-        None => false,
-    }
+    let trap = match raw.error.downcast_ref::<Trap>() {
+        Some(trap) => trap,
+        None => return false,
+    };
+    *code = match trap {
+        Trap::StackOverflow => 0,
+        Trap::MemoryOutOfBounds => 1,
+        Trap::HeapMisaligned => 2,
+        Trap::TableOutOfBounds => 3,
+        Trap::IndirectCallToNull => 4,
+        Trap::BadSignature => 5,
+        Trap::IntegerOverflow => 6,
+        Trap::IntegerDivisionByZero => 7,
+        Trap::BadConversionToInteger => 8,
+        Trap::UnreachableCodeReached => 9,
+        Trap::Interrupt => 10,
+        _ => unreachable!(),
+    };
+    true
 }
 
 #[no_mangle]
 pub extern "C" fn wasmtime_trap_exit_status(raw: &wasm_trap_t, status: &mut i32) -> bool {
-    match raw.trap.i32_exit_status() {
-        Some(i) => {
-            *status = i;
-            true
-        }
-        None => false,
+    #[cfg(feature = "wasi")]
+    if let Some(exit) = raw.error.downcast_ref::<wasmtime_wasi::I32Exit>() {
+        *status = exit.0;
+        return true;
     }
+
+    // Squash unused warnings in wasi-disabled builds.
+    drop((raw, status));
+
+    false
 }
 
 #[no_mangle]
-pub extern "C" fn wasm_frame_func_index(frame: &wasm_frame_t) -> u32 {
-    frame.trap.trace().expect("backtraces are always enabled")[frame.idx].func_index()
+pub extern "C" fn wasm_frame_func_index(frame: &wasm_frame_t<'_>) -> u32 {
+    frame.trace.frames()[frame.idx].func_index()
 }
 
 #[no_mangle]
-pub extern "C" fn wasmtime_frame_func_name(frame: &wasm_frame_t) -> Option<&wasm_name_t> {
+pub extern "C" fn wasmtime_frame_func_name<'a>(
+    frame: &'a wasm_frame_t<'_>,
+) -> Option<&'a wasm_name_t> {
     frame
         .func_name
         .get_or_init(|| {
-            frame.trap.trace().expect("backtraces are always enabled")[frame.idx]
+            frame.trace.frames()[frame.idx]
                 .func_name()
                 .map(|s| wasm_name_t::from(s.to_string().into_bytes()))
         })
@@ -144,11 +168,13 @@ pub extern "C" fn wasmtime_frame_func_name(frame: &wasm_frame_t) -> Option<&wasm
 }
 
 #[no_mangle]
-pub extern "C" fn wasmtime_frame_module_name(frame: &wasm_frame_t) -> Option<&wasm_name_t> {
+pub extern "C" fn wasmtime_frame_module_name<'a>(
+    frame: &'a wasm_frame_t<'_>,
+) -> Option<&'a wasm_name_t> {
     frame
         .module_name
         .get_or_init(|| {
-            frame.trap.trace().expect("backtraces are always enabled")[frame.idx]
+            frame.trace.frames()[frame.idx]
                 .module_name()
                 .map(|s| wasm_name_t::from(s.to_string().into_bytes()))
         })
@@ -156,25 +182,25 @@ pub extern "C" fn wasmtime_frame_module_name(frame: &wasm_frame_t) -> Option<&wa
 }
 
 #[no_mangle]
-pub extern "C" fn wasm_frame_func_offset(frame: &wasm_frame_t) -> usize {
-    frame.trap.trace().expect("backtraces are always enabled")[frame.idx]
+pub extern "C" fn wasm_frame_func_offset(frame: &wasm_frame_t<'_>) -> usize {
+    frame.trace.frames()[frame.idx]
         .func_offset()
         .unwrap_or(usize::MAX)
 }
 
 #[no_mangle]
-pub extern "C" fn wasm_frame_instance(_arg1: *const wasm_frame_t) -> *mut wasm_instance_t {
+pub extern "C" fn wasm_frame_instance(_arg1: *const wasm_frame_t<'_>) -> *mut wasm_instance_t {
     unimplemented!("wasm_frame_instance")
 }
 
 #[no_mangle]
-pub extern "C" fn wasm_frame_module_offset(frame: &wasm_frame_t) -> usize {
-    frame.trap.trace().expect("backtraces are always enabled")[frame.idx]
+pub extern "C" fn wasm_frame_module_offset(frame: &wasm_frame_t<'_>) -> usize {
+    frame.trace.frames()[frame.idx]
         .module_offset()
         .unwrap_or(usize::MAX)
 }
 
 #[no_mangle]
-pub extern "C" fn wasm_frame_copy(frame: &wasm_frame_t) -> Box<wasm_frame_t> {
+pub extern "C" fn wasm_frame_copy<'a>(frame: &wasm_frame_t<'a>) -> Box<wasm_frame_t<'a>> {
     Box::new(frame.clone())
 }

crates/c-api/src/vec.rs

@@ -19,7 +19,7 @@ impl wasm_name_t {
 macro_rules! declare_vecs {
     (
         $((
-            name: $name:ident,
+            name: $name:ident $(<$lt:tt>)?,
             ty: $elem_ty:ty,
             new: $new:ident,
             empty: $empty:ident,
@@ -29,12 +29,12 @@ macro_rules! declare_vecs {
         ))*
     ) => {$(
         #[repr(C)]
-        pub struct $name {
+        pub struct $name $(<$lt>)? {
             size: usize,
             data: *mut $elem_ty,
         }
 
-        impl $name {
+        impl$(<$lt>)? $name $(<$lt>)? {
             pub fn set_buffer(&mut self, buffer: Vec<$elem_ty>) {
                 let mut vec = buffer.into_boxed_slice();
                 self.size = vec.len();
@@ -79,13 +79,13 @@ macro_rules! declare_vecs {
             }
         }
 
-        impl Clone for $name {
+        impl$(<$lt>)? Clone for $name $(<$lt>)? {
             fn clone(&self) -> Self {
                 self.as_slice().to_vec().into()
             }
         }
 
-        impl From<Vec<$elem_ty>> for $name {
+        impl$(<$lt>)? From<Vec<$elem_ty>> for $name $(<$lt>)? {
             fn from(vec: Vec<$elem_ty>) -> Self {
                 let mut vec = vec.into_boxed_slice();
                 let result = $name {
@@ -97,7 +97,7 @@ macro_rules! declare_vecs {
             }
         }
 
-        impl Drop for $name {
+        impl$(<$lt>)? Drop for $name $(<$lt>)? {
             fn drop(&mut self) {
                 drop(self.take());
             }
@@ -115,8 +115,8 @@ macro_rules! declare_vecs {
         }
 
         #[no_mangle]
-        pub unsafe extern "C" fn $new(
-            out: &mut $name,
+        pub unsafe extern "C" fn $new $(<$lt>)? (
+            out: &mut $name $(<$lt>)?,
             size: usize,
             ptr: *const $elem_ty,
         ) {
@@ -125,12 +125,15 @@ macro_rules! declare_vecs {
         }
 
         #[no_mangle]
-        pub extern "C" fn $copy(out: &mut $name, src: &$name) {
+        pub extern "C" fn $copy $(<$lt>)? (
+            out: &mut $name $(<$lt>)?,
+            src: &$name $(<$lt>)?,
+        ) {
             out.set_buffer(src.as_slice().to_vec());
         }
 
         #[no_mangle]
-        pub extern "C" fn $delete(out: &mut $name) {
+        pub extern "C" fn $delete $(<$lt>)? (out: &mut $name $(<$lt>)?) {
             out.take();
         }
     )*};
@@ -228,8 +231,8 @@ declare_vecs! {
         delete: wasm_val_vec_delete,
     )
     (
-        name: wasm_frame_vec_t,
-        ty: Option<Box<wasm_frame_t>>,
+        name: wasm_frame_vec_t<'a>,
+        ty: Option<Box<wasm_frame_t<'a>>>,
         new: wasm_frame_vec_new,
         empty: wasm_frame_vec_new_empty,
         uninit: wasm_frame_vec_new_uninitialized,