Add *_unchecked variants of Func APIs for the C API (#3350)

* Add `*_unchecked` variants of `Func` APIs for the C API

This commit is what is hopefully going to be my last installment within
the saga of optimizing function calls in/out of WebAssembly modules in
the C API. This is yet another alternative approach to #3345 (sorry) but
also contains everything necessary to make the C API fast. As in #3345
the general idea is just moving checks out of the call path in the same
style of `TypedFunc`.

This new strategy takes inspiration from previously learned attempts
effectively "just" exposes how we previously passed `*mut u128` through
trampolines for arguments/results. This storage format is formalized
through a new `ValRaw` union that is exposed from the `wasmtime` crate.
By doing this it made it relatively easy to expose two new APIs:

* `Func::new_unchecked`
* `Func::call_unchecked`

These are the same as their checked equivalents except that they're
`unsafe` and they work with `*mut ValRaw` rather than safe slices of
`Val`. Working with these eschews type checks and such and requires
callers/embedders to do the right thing.

These two new functions are then exposed via the C API with new
functions, enabling C to have a fast-path of calling/defining functions.
This fast path is akin to `Func::wrap` in Rust, although that API can't
be built in C due to C not having generics in the same way that Rust
has.

For some benchmarks, the benchmarks here are:

* `nop` - Call a wasm function from the host that does nothing and
  returns nothing.
* `i64` - Call a wasm function from the host, the wasm function calls a
  host function, and the host function returns an `i64` all the way out to
  the original caller.
* `many` - Call a wasm function from the host, the wasm calls
   host function with 5 `i32` parameters, and then an `i64` result is
   returned back to the original host
* `i64` host - just the overhead of the wasm calling the host, so the
  wasm calls the host function in a loop.
* `many` host - same as `i64` host, but calling the `many` host function.

All numbers in this table are in nanoseconds, and this is just one
measurement as well so there's bound to be some variation in the precise
numbers here.

| Name      | Rust | C (before) | C (after) |
|-----------|------|------------|-----------|
| nop       | 19   | 112        | 25        |
| i64       | 22   | 207        | 32        |
| many      | 27   | 189        | 34        |
| i64 host  | 2    | 38         | 5         |
| many host | 7    | 75         | 8         |

The main conclusion here is that the C API is significantly faster than
before when using the `*_unchecked` variants of APIs. The Rust
implementation is still the ceiling (or floor I guess?) for performance
The main reason that C is slower than Rust is that a little bit more has
to travel through memory where on the Rust side of things we can
monomorphize and inline a bit more to get rid of that. Overall though
the costs are way way down from where they were originally and I don't
plan on doing a whole lot more myself at this time. There's various
things we theoretically could do I've considered but implementation-wise
I think they'll be much more weighty.

* Tweak `wasmtime_externref_t` API comments

crates/c-api/include/wasmtime/func.h

@@ -87,6 +87,75 @@ WASM_API_EXTERN void wasmtime_func_new(
   wasmtime_func_t *ret
 );
 
+/**
+ * \brief Callback signature for #wasmtime_func_new_unchecked.
+ *
+ * This is the function signature for host functions that can be made accessible
+ * to WebAssembly. The arguments to this function are:
+ *
+ * \param env user-provided argument passed to #wasmtime_func_new_unchecked
+ * \param caller a temporary object that can only be used during this function
+ *        call. Used to acquire #wasmtime_context_t or caller's state
+ * \param args_and_results storage space for both the parameters to the
+ *        function as well as the results of the function. The size of this
+ *        array depends on the function type that the host function is created
+ *        with, but it will be the maximum of the number of parameters and
+ *        number of results.
+ *
+ * This callback can optionally return a #wasm_trap_t indicating that a trap
+ * should be raised in WebAssembly. It's expected that in this case the caller
+ * relinquishes ownership of the trap and it is passed back to the engine.
+ *
+ * This differs from #wasmtime_func_callback_t in that the payload of
+ * `args_and_results` does not have type information, nor does it have sizing
+ * information. This is especially unsafe because it's only valid within the
+ * particular #wasm_functype_t that the function was created with. The onus is
+ * on the embedder to ensure that `args_and_results` are all read correctly
+ * for parameters and all written for results within the execution of a
+ * function.
+ *
+ * Parameters will be listed starting at index 0 in the `args_and_results`
+ * array. Results are also written starting at index 0, which will overwrite
+ * the arguments.
+ */
+typedef wasm_trap_t* (*wasmtime_func_unchecked_callback_t)(
+    void *env,
+    wasmtime_caller_t* caller,
+    wasmtime_val_raw_t *args_and_results);
+
+/**
+ * \brief Creates a new host function in the same manner of #wasmtime_func_new,
+ *        but the function-to-call has no type information available at runtime.
+ *
+ * This function is very similar to #wasmtime_func_new. The difference is that
+ * this version is "more unsafe" in that when the host callback is invoked there
+ * is no type information and no checks that the right types of values are
+ * produced. The onus is on the consumer of this API to ensure that all
+ * invariants are upheld such as:
+ *
+ * * The host callback reads parameters correctly and interprets their types
+ *   correctly.
+ * * If a trap doesn't happen then all results are written to the results
+ *   pointer. All results must have the correct type.
+ * * Types such as `funcref` cannot cross stores.
+ * * Types such as `externref` have valid reference counts.
+ *
+ * It's generally only recommended to use this if your application can wrap
+ * this in a safe embedding. This should not be frequently used due to the
+ * number of invariants that must be upheld on the wasm<->host boundary. On the
+ * upside, though, this flavor of host function will be faster to call than
+ * those created by #wasmtime_func_new (hence the reason for this function's
+ * existence).
+ */
+WASM_API_EXTERN void wasmtime_func_new_unchecked(
+  wasmtime_context_t *store,
+  const wasm_functype_t* type,
+  wasmtime_func_unchecked_callback_t callback,
+  void *env,
+  void (*finalizer)(void*),
+  wasmtime_func_t *ret
+);
+
 /**
  * \brief Returns the type of the function specified
  *
@@ -142,6 +211,39 @@ WASM_API_EXTERN wasmtime_error_t *wasmtime_func_call(
     wasm_trap_t **trap
 );
 
+/**
+ * \brief Call a WebAssembly function in an "unchecked" fashion.
+ *
+ * This function is similar to #wasmtime_func_call except that there is no type
+ * information provided with the arguments (or sizing information). Consequently
+ * this is less safe to call since it's up to the caller to ensure that `args`
+ * has an appropriate size and all the parameters are configured with their
+ * appropriate values/types. Additionally all the results must be interpreted
+ * correctly if this function returns successfully.
+ *
+ * Parameters must be specified starting at index 0 in the `args_and_results`
+ * array. Results are written starting at index 0, which will overwrite
+ * the arguments.
+ *
+ * Callers must ensure that various correctness variants are upheld when this
+ * API is called such as:
+ *
+ * * The `args_and_results` pointer has enough space to hold all the parameters
+ *   and all the results (but not at the same time).
+ * * Parameters must all be configured as if they were the correct type.
+ * * Values such as `externref` and `funcref` are valid within the store being
+ *   called.
+ *
+ * When in doubt it's much safer to call #wasmtime_func_call. This function is
+ * faster than that function, but the tradeoff is that embeddings must uphold
+ * more invariants rather than relying on Wasmtime to check them for you.
+ */
+WASM_API_EXTERN wasm_trap_t *wasmtime_func_call_unchecked(
+    wasmtime_context_t *store,
+    const wasmtime_func_t *func,
+    wasmtime_val_raw_t *args_and_results
+);
+
 /**
  * \brief Loads a #wasmtime_extern_t from the caller's context
  *
@@ -172,6 +274,32 @@ WASM_API_EXTERN bool wasmtime_caller_export_get(
  */
 WASM_API_EXTERN wasmtime_context_t* wasmtime_caller_context(wasmtime_caller_t* caller);
 
+/**
+ * \brief Converts a `raw` nonzero `funcref` value from #wasmtime_val_raw_t
+ * into a #wasmtime_func_t.
+ *
+ * This function can be used to interpret nonzero values of the `funcref` field
+ * of the #wasmtime_val_raw_t structure. It is assumed that `raw` does not have
+ * a value of 0, or otherwise the program will abort.
+ *
+ * Note that this function is unchecked and unsafe. It's only safe to pass
+ * values learned from #wasmtime_val_raw_t with the same corresponding
+ * #wasmtime_context_t that they were produced from. Providing arbitrary values
+ * to `raw` here or cross-context values with `context` is UB.
+ */
+WASM_API_EXTERN void wasmtime_func_from_raw(
+    wasmtime_context_t* context,
+    size_t raw,
+    wasmtime_func_t *ret);
+
+/**
+ * \brief Converts a `func`  which belongs to `context` into a `usize`
+ * parameter that is suitable for insertion into a #wasmtime_val_raw_t.
+ */
+WASM_API_EXTERN size_t wasmtime_func_to_raw(
+    wasmtime_context_t* context,
+    const wasmtime_func_t *func);
+
 #ifdef __cplusplus
 }  // extern "C"
 #endif

crates/c-api/include/wasmtime/linker.h

@@ -102,6 +102,8 @@ WASM_API_EXTERN wasmtime_error_t* wasmtime_linker_define(
  * Note that this function does not create a #wasmtime_func_t. This creates a
  * store-independent function within the linker, allowing this function
  * definition to be used with multiple stores.
+ *
+ * For more information about host callbacks see #wasmtime_func_new.
  */
 WASM_API_EXTERN wasmtime_error_t* wasmtime_linker_define_func(
     wasmtime_linker_t *linker,
@@ -115,6 +117,27 @@ WASM_API_EXTERN wasmtime_error_t* wasmtime_linker_define_func(
     void (*finalizer)(void*)
 );
 
+/**
+ * \brief Defines a new function in this linker.
+ *
+ * This is the same as #wasmtime_linker_define_func except that it's the analog
+ * of #wasmtime_func_new_unchecked instead of #wasmtime_func_new. Be sure to
+ * consult the documentation of #wasmtime_linker_define_func for argument
+ * information as well as #wasmtime_func_new_unchecked for why this is an
+ * unsafe API.
+ */
+WASM_API_EXTERN wasmtime_error_t* wasmtime_linker_define_func_unchecked(
+    wasmtime_linker_t *linker,
+    const char *module,
+    size_t module_len,
+    const char *name,
+    size_t name_len,
+    const wasm_functype_t *ty,
+    wasmtime_func_unchecked_callback_t cb,
+    void *data,
+    void (*finalizer)(void*)
+);
+
 /**
  * \brief Defines WASI functions in this linker.
  *

crates/c-api/include/wasmtime/val.h

@@ -63,6 +63,29 @@ WASM_API_EXTERN wasmtime_externref_t *wasmtime_externref_clone(wasmtime_externre
  */
 WASM_API_EXTERN void wasmtime_externref_delete(wasmtime_externref_t *ref);
 
+/**
+ * \brief Converts a raw `externref` value coming from #wasmtime_val_raw_t into
+ * a #wasmtime_externref_t.
+ *
+ * Note that the returned #wasmtime_externref_t is an owned value that must be
+ * deleted via #wasmtime_externref_delete by the caller if it is non-null.
+ */
+WASM_API_EXTERN wasmtime_externref_t *wasmtime_externref_from_raw(wasmtime_context_t *context, size_t raw);
+
+/**
+ * \brief Converts a #wasmtime_externref_t to a raw value suitable for storing
+ * into a #wasmtime_val_raw_t.
+ *
+ * Note that the returned underlying value is not tracked by Wasmtime's garbage
+ * collector until it enters WebAssembly. This means that a GC may release the
+ * context's reference to the raw value, making the raw value invalid within the
+ * context of the store. Do not perform a GC between calling this function and
+ * passing it to WebAssembly.
+ */
+WASM_API_EXTERN size_t wasmtime_externref_to_raw(
+    wasmtime_context_t *context,
+    const wasmtime_externref_t *ref);
+
 /// \brief Discriminant stored in #wasmtime_val::kind
 typedef uint8_t wasmtime_valkind_t;
 /// \brief Value of #wasmtime_valkind_t meaning that #wasmtime_val_t is an i32
@@ -117,6 +140,43 @@ typedef union wasmtime_valunion {
   wasmtime_v128 v128;
 } wasmtime_valunion_t;
 
+/**
+ * \typedef wasmtime_val_raw_t
+ * \brief Convenience alias for #wasmtime_val_raw
+ *
+ * \union wasmtime_val_raw
+ * \brief Container for possible wasm values.
+ *
+ * This type is used on conjunction with #wasmtime_func_new_unchecked as well
+ * as #wasmtime_func_call_unchecked. Instances of this type do not have type
+ * information associated with them, it's up to the embedder to figure out
+ * how to interpret the bits contained within, often using some other channel
+ * to determine the type.
+ */
+typedef union wasmtime_val_raw {
+  /// Field for when this val is a WebAssembly `i32` value.
+  int32_t i32;
+  /// Field for when this val is a WebAssembly `i64` value.
+  int64_t i64;
+  /// Field for when this val is a WebAssembly `f32` value.
+  float32_t f32;
+  /// Field for when this val is a WebAssembly `f64` value.
+  float64_t f64;
+  /// Field for when this val is a WebAssembly `v128` value.
+  wasmtime_v128 v128;
+  /// Field for when this val is a WebAssembly `funcref` value.
+  ///
+  /// If this is set to 0 then it's a null funcref, otherwise this must be
+  /// passed to `wasmtime_func_from_raw` to determine the `wasmtime_func_t`.
+  size_t funcref;
+  /// Field for when this val is a WebAssembly `externref` value.
+  ///
+  /// If this is set to 0 then it's a null externref, otherwise this must be
+  /// passed to `wasmtime_externref_from_raw` to determine the
+  /// `wasmtime_externref_t`.
+  size_t externref;
+} wasmtime_val_raw_t;
+
 /**
  * \typedef wasmtime_val_t
  * \brief Convenience alias for #wasmtime_val_t

crates/c-api/src/func.rs

@@ -8,7 +8,7 @@ use std::mem::{self, MaybeUninit};
 use std::panic::{self, AssertUnwindSafe};
 use std::ptr;
 use std::str;
-use wasmtime::{AsContextMut, Caller, Extern, Func, Trap, Val};
+use wasmtime::{AsContextMut, Caller, Extern, Func, Trap, Val, ValRaw};
 
 #[derive(Clone)]
 #[repr(transparent)]
@@ -208,6 +208,9 @@ pub type wasmtime_func_callback_t = extern "C" fn(
     usize,
 ) -> Option<Box<wasm_trap_t>>;
 
+pub type wasmtime_func_unchecked_callback_t =
+    extern "C" fn(*mut c_void, *mut wasmtime_caller_t, *mut ValRaw) -> Option<Box<wasm_trap_t>>;
+
 #[no_mangle]
 pub unsafe extern "C" fn wasmtime_func_new(
     store: CStoreContextMut<'_>,
@@ -271,6 +274,35 @@ pub(crate) unsafe fn c_callback_to_rust_fn(
     }
 }
 
+#[no_mangle]
+pub unsafe extern "C" fn wasmtime_func_new_unchecked(
+    store: CStoreContextMut<'_>,
+    ty: &wasm_functype_t,
+    callback: wasmtime_func_unchecked_callback_t,
+    data: *mut c_void,
+    finalizer: Option<extern "C" fn(*mut std::ffi::c_void)>,
+    func: &mut Func,
+) {
+    let ty = ty.ty().ty.clone();
+    let cb = c_unchecked_callback_to_rust_fn(callback, data, finalizer);
+    *func = Func::new_unchecked(store, ty, cb);
+}
+
+pub(crate) unsafe fn c_unchecked_callback_to_rust_fn(
+    callback: wasmtime_func_unchecked_callback_t,
+    data: *mut c_void,
+    finalizer: Option<extern "C" fn(*mut std::ffi::c_void)>,
+) -> impl Fn(Caller<'_, crate::StoreData>, *mut ValRaw) -> Result<(), Trap> {
+    let foreign = crate::ForeignData { data, finalizer };
+    move |caller, values| {
+        let mut caller = wasmtime_caller_t { caller };
+        match callback(foreign.data, &mut caller, values) {
+            None => Ok(()),
+            Some(trap) => Err(trap.trap),
+        }
+    }
+}
+
 #[no_mangle]
 pub unsafe extern "C" fn wasmtime_func_call(
     mut store: CStoreContextMut<'_>,
@@ -329,6 +361,18 @@ pub unsafe extern "C" fn wasmtime_func_call(
     }
 }
 
+#[no_mangle]
+pub unsafe extern "C" fn wasmtime_func_call_unchecked(
+    store: CStoreContextMut<'_>,
+    func: &Func,
+    args_and_results: *mut ValRaw,
+) -> *mut wasm_trap_t {
+    match func.call_unchecked(store, args_and_results) {
+        Ok(()) => ptr::null_mut(),
+        Err(trap) => Box::into_raw(Box::new(wasm_trap_t::new(trap))),
+    }
+}
+
 #[no_mangle]
 pub extern "C" fn wasmtime_func_type(
     store: CStoreContext<'_>,
@@ -362,3 +406,17 @@ pub unsafe extern "C" fn wasmtime_caller_export_get(
     crate::initialize(item, which.into());
     true
 }
+
+#[no_mangle]
+pub unsafe extern "C" fn wasmtime_func_from_raw(
+    store: CStoreContextMut<'_>,
+    raw: usize,
+    func: &mut Func,
+) {
+    *func = Func::from_raw(store, raw).unwrap();
+}
+
+#[no_mangle]
+pub unsafe extern "C" fn wasmtime_func_to_raw(store: CStoreContextMut<'_>, func: &Func) -> usize {
+    func.to_raw(store)
+}

crates/c-api/src/linker.rs

@@ -1,7 +1,6 @@
-use crate::func::c_callback_to_rust_fn;
 use crate::{
     bad_utf8, handle_result, wasm_engine_t, wasm_functype_t, wasm_trap_t, wasmtime_error_t,
-    wasmtime_extern_t, wasmtime_func_callback_t, wasmtime_module_t, CStoreContextMut,
+    wasmtime_extern_t, wasmtime_module_t, CStoreContextMut,
 };
 use std::ffi::c_void;
 use std::mem::MaybeUninit;
@@ -64,17 +63,39 @@ pub unsafe extern "C" fn wasmtime_linker_define_func(
     name: *const u8,
     name_len: usize,
     ty: &wasm_functype_t,
-    callback: wasmtime_func_callback_t,
+    callback: crate::wasmtime_func_callback_t,
     data: *mut c_void,
     finalizer: Option<extern "C" fn(*mut std::ffi::c_void)>,
 ) -> Option<Box<wasmtime_error_t>> {
     let ty = ty.ty().ty.clone();
     let module = to_str!(module, module_len);
     let name = to_str!(name, name_len);
-    let cb = c_callback_to_rust_fn(callback, data, finalizer);
+    let cb = crate::func::c_callback_to_rust_fn(callback, data, finalizer);
     handle_result(linker.linker.func_new(module, name, ty, cb), |_linker| ())
 }
 
+#[no_mangle]
+pub unsafe extern "C" fn wasmtime_linker_define_func_unchecked(
+    linker: &mut wasmtime_linker_t,
+    module: *const u8,
+    module_len: usize,
+    name: *const u8,
+    name_len: usize,
+    ty: &wasm_functype_t,
+    callback: crate::wasmtime_func_unchecked_callback_t,
+    data: *mut c_void,
+    finalizer: Option<extern "C" fn(*mut std::ffi::c_void)>,
+) -> Option<Box<wasmtime_error_t>> {
+    let ty = ty.ty().ty.clone();
+    let module = to_str!(module, module_len);
+    let name = to_str!(name, name_len);
+    let cb = crate::func::c_unchecked_callback_to_rust_fn(callback, data, finalizer);
+    handle_result(
+        linker.linker.func_new_unchecked(module, name, ty, cb),
+        |_linker| (),
+    )
+}
+
 #[cfg(feature = "wasi")]
 #[no_mangle]
 pub extern "C" fn wasmtime_linker_define_wasi(

crates/c-api/src/val.rs

@@ -1,5 +1,8 @@
 use crate::r#ref::{ref_to_val, WasmRefInner};
-use crate::{from_valtype, into_valtype, wasm_ref_t, wasm_valkind_t, wasmtime_valkind_t, WASM_I32};
+use crate::{
+    from_valtype, into_valtype, wasm_ref_t, wasm_valkind_t, wasmtime_valkind_t, CStoreContextMut,
+    WASM_I32,
+};
 use std::ffi::c_void;
 use std::mem::{self, ManuallyDrop, MaybeUninit};
 use std::ptr;
@@ -288,3 +291,22 @@ pub extern "C" fn wasmtime_externref_clone(externref: ManuallyDrop<ExternRef>) -
 
 #[no_mangle]
 pub extern "C" fn wasmtime_externref_delete(_val: Option<ExternRef>) {}
+
+#[no_mangle]
+pub unsafe extern "C" fn wasmtime_externref_to_raw(
+    cx: CStoreContextMut<'_>,
+    val: Option<ManuallyDrop<ExternRef>>,
+) -> usize {
+    match val {
+        Some(ptr) => ptr.to_raw(cx),
+        None => 0,
+    }
+}
+
+#[no_mangle]
+pub unsafe extern "C" fn wasmtime_externref_from_raw(
+    _cx: CStoreContextMut<'_>,
+    val: usize,
+) -> Option<ExternRef> {
+    ExternRef::from_raw(val)
+}