T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Implement canon lower of a canon lift function in the same component (#4347)

Browse Source 
* Implement `canon lower` of a `canon lift` function in the same component

This commit implements the "degenerate" logic for implementing a
function within a component that is lifted and then immediately lowered
again. In this situation the lowered function will immediately generate
a trap and doesn't need to implement anything else.

The implementation in this commit is somewhat heavyweight but I think is
probably justified moreso in future additions to the component model
rather than what exactly is here right now. It's not expected that this
"always trap" functionality will really be used all that often since it
would generally mean a buggy component, but the functionality plumbed
through here is hopefully going to be useful for implementing
component-to-component adapter trampolines.

Specifically this commit implements a strategy where the `canon.lower`'d
function is generated by Cranelift and simply has a single trap
instruction when called, doing nothing else. The main complexity comes
from juggling around all the data associated with these functions,
primarily plumbing through the traps into the `ModuleRegistry` to
ensure that the global `is_wasm_trap_pc` function returns `true` and at
runtime when we lookup information about the trap it's all readily
available (e.g. translating the trapping pc to a `TrapCode`).

* Fix non-component build

* Fix some offset calculations

* Only create one "always trap" per signature

Use an internal map to deduplicate during compilation.
This commit is contained in:
Alex Crichton
2022-06-29 11:35:37 -05:00
committed by GitHub
parent 22fb3ecbbf
commit f0278c5db7
16 changed files with 664 additions and 164 deletions
Download Patch File Download Diff File Expand all files Collapse all files

33
crates/environ/src/component/compiler.rs
View File

@@ -1,5 +1,7 @@
use crate::component::{Component, ComponentTypes, LowerImport, LoweredIndex};
use crate::{PrimaryMap, SignatureIndex, Trampoline};
use crate::component::{
Component, ComponentTypes, LowerImport, LoweredIndex, RuntimeAlwaysTrapIndex,
};
use crate::{PrimaryMap, SignatureIndex, Trampoline, WasmFuncType};
use anyhow::Result;
use object::write::Object;
use serde::{Deserialize, Serialize};
@@ -16,6 +18,19 @@ pub struct LoweringInfo {
pub length: u32,
}
/// Description of an "always trap" function generated by
/// `ComponentCompiler::compile_always_trap`.
#[derive(Serialize, Deserialize)]
pub struct AlwaysTrapInfo {
/// The byte offset from the start of the text section where this trampoline
/// starts.
pub start: u32,
/// The byte length of this trampoline's function body.
pub length: u32,
/// The offset from `start` of where the trapping instruction is located.
pub trap_offset: u32,
}
/// Compilation support necessary for components.
pub trait ComponentCompiler: Send + Sync {
/// Creates a trampoline for a `canon.lower`'d host function.
@@ -42,6 +57,13 @@ pub trait ComponentCompiler: Send + Sync {
types: &ComponentTypes,
) -> Result<Box<dyn Any + Send>>;
/// Creates a function which will always trap that has the `ty` specified.
///
/// This will create a small trampoline whose only purpose is to generate a
/// trap at runtime. This is used to implement the degenerate case of a
/// `canon lift`'d function immediately being `canon lower`'d.
fn compile_always_trap(&self, ty: &WasmFuncType) -> Result<Box<dyn Any + Send>>;
/// Emits the `lowerings` and `trampolines` specified into the in-progress
/// ELF object specified by `obj`.
///
@@ -53,7 +75,12 @@ pub trait ComponentCompiler: Send + Sync {
fn emit_obj(
&self,
lowerings: PrimaryMap<LoweredIndex, Box<dyn Any + Send>>,
always_trap: PrimaryMap<RuntimeAlwaysTrapIndex, Box<dyn Any + Send>>,
tramplines: Vec<(SignatureIndex, Box<dyn Any + Send>)>,
obj: &mut Object<'static>,
) -> Result<(PrimaryMap<LoweredIndex, LoweringInfo>, Vec<Trampoline>)>;
) -> Result<(
PrimaryMap<LoweredIndex, LoweringInfo>,
PrimaryMap<RuntimeAlwaysTrapIndex, AlwaysTrapInfo>,
Vec<Trampoline>,
)>;
}

25
crates/environ/src/component/info.rs
View File

@@ -143,6 +143,10 @@ pub struct Component {
/// The number of modules that are required to be saved within an instance
/// at runtime, or effectively the number of exported modules.
pub num_runtime_modules: u32,
/// The number of functions which "always trap" used to implement
/// `canon.lower` of `canon.lift`'d functions within the same component.
pub num_always_trap: u32,
}
/// GlobalInitializer instructions to get processed when instantiating a component
@@ -173,6 +177,12 @@ pub enum GlobalInitializer {
/// pointer the trampoline calls, and the canonical ABI options.
LowerImport(LowerImport),
/// A core wasm function was "generated" via `canon lower` of a function
/// that was `canon lift`'d in the same component, meaning that the function
/// always traps. This is recorded within the `VMComponentContext` as a new
/// `VMCallerCheckedAnyfunc` that's available for use.
AlwaysTrap(AlwaysTrap),
/// A core wasm linear memory is going to be saved into the
/// `VMComponentContext`.
///
@@ -272,6 +282,17 @@ pub struct LowerImport {
pub options: CanonicalOptions,
}
/// Description of what to initialize when a `GlobalInitializer::AlwaysTrap` is
/// encountered.
#[derive(Debug, Serialize, Deserialize)]
pub struct AlwaysTrap {
/// The index of the function that is being initialized in the
/// `VMComponentContext`.
pub index: RuntimeAlwaysTrapIndex,
/// The core wasm signature of the function that's inserted.
pub canonical_abi: SignatureIndex,
}
/// Definition of a core wasm item and where it can come from within a
/// component.
///
@@ -288,6 +309,10 @@ pub enum CoreDef {
/// that this `LoweredIndex` corresponds to the nth
/// `GlobalInitializer::LowerImport` instruction.
Lowered(LoweredIndex),
/// This is used to represent a degenerate case of where a `canon lift`'d
/// function is immediately `canon lower`'d in the same instance. Such a
/// function always traps at runtime.
AlwaysTrap(RuntimeAlwaysTrapIndex),
}
impl<T> From<CoreExport<T>> for CoreDef

122
crates/environ/src/component/translate/inline.rs
View File

@@ -46,7 +46,7 @@
//! final `Component`.
use crate::component::translate::*;
use crate::{ModuleTranslation, PrimaryMap};
use crate::{ModuleTranslation, PrimaryMap, SignatureIndex};
use indexmap::IndexMap;
pub(super) fn run(
@@ -64,6 +64,7 @@ pub(super) fn run(
runtime_realloc_interner: Default::default(),
runtime_post_return_interner: Default::default(),
runtime_memory_interner: Default::default(),
runtime_always_trap_interner: Default::default(),
};
// The initial arguments to the root component are all host imports. This
@@ -194,6 +195,7 @@ struct Inliner<'a> {
runtime_realloc_interner: HashMap<CoreDef, RuntimeReallocIndex>,
runtime_post_return_interner: HashMap<CoreDef, RuntimePostReturnIndex>,
runtime_memory_interner: HashMap<CoreExport<MemoryIndex>, RuntimeMemoryIndex>,
runtime_always_trap_interner: HashMap<SignatureIndex, RuntimeAlwaysTrapIndex>,
}
/// A "stack frame" as part of the inlining process, or the progress through
@@ -443,12 +445,6 @@ impl<'a> Inliner<'a> {
//
// NB: at this time only lowered imported functions are supported.
Lower(func, options) => {
// Assign this lowering a unique index and determine the core
// wasm function index we're defining.
let index = LoweredIndex::from_u32(self.result.num_lowerings);
self.result.num_lowerings += 1;
let func_index = frame.funcs.push(CoreDef::Lowered(index));
// Use the type information from `wasmparser` to lookup the core
// wasm function signature of the lowered function. This avoids
// us having to reimplement the
@@ -461,20 +457,22 @@ impl<'a> Inliner<'a> {
.types
.as_ref()
.unwrap()
.function_at(func_index.as_u32())
.function_at(frame.funcs.next_key().as_u32())
.expect("should be in-bounds");
let canonical_abi = self
.types
.module_types_builder()
.wasm_func_type(lowered_function_type.clone().try_into()?);
let options = self.canonical_options(frame, options);
match &frame.component_funcs[*func] {
let options_lower = self.canonical_options(frame, options);
let func = match &frame.component_funcs[*func] {
// If this component function was originally a host import
// then this is a lowered host function which needs a
// trampoline to enter WebAssembly. That's recorded here
// with all relevant information.
ComponentFuncDef::Import(path) => {
let index = LoweredIndex::from_u32(self.result.num_lowerings);
self.result.num_lowerings += 1;
let import = self.runtime_import(path);
self.result
.initializers
@@ -482,35 +480,91 @@ impl<'a> Inliner<'a> {
canonical_abi,
import,
index,
options,
options: options_lower,
}));
CoreDef::Lowered(index)
}
// TODO: Lowering a lift function could mean one of two
// things:
// This case handles when a lifted function is later
// lowered, and both the lowering and the lifting are
// happening within the same component instance.
//
// * This could mean that a "fused adapter" was just
// identified. If the lifted function here comes from a
// different component than we're lowering into then we
// have identified the fusion location of two components
// talking to each other. Metadata needs to be recorded
// here about the fusion to get something generated by
// Cranelift later on.
// In this situation if the `canon.lower`'d function is
// called then it immediately sets `may_enter` to `false`.
// When calling the callee, however, that's `canon.lift`
// which immediately traps if `may_enter` is `false`. That
// means that this pairing of functions creates a function
// that always traps.
//
// * Otherwise if the lifted function is in the same
// component that we're lowering into then that means
// something "funky" is happening. This needs to be
// carefully implemented with respect to the
// may_{enter,leave} flags as specified with the canonical
// ABI. The careful consideration for how to do this has
// not yet happened.
// When closely reading the spec though the precise trap
// that comes out can be somewhat variable. Technically the
// function yielded here is one that should validate the
// arguments by lifting them, and then trap. This means that
// the trap could be different depending on whether all
// arguments are valid for now. This was discussed in
// WebAssembly/component-model#51 somewhat and the
// conclusion was that we can probably get away with "always
// trap" here.
//
// In general this is almost certainly going to require some
// new variant of `GlobalInitializer` in one form or another.
ComponentFuncDef::Lifted { .. } => {
// The `CoreDef::AlwaysTrap` variant here is used to
// indicate that this function is valid but if something
// actually calls it then it just generates a trap
// immediately.
ComponentFuncDef::Lifted {
options: options_lift,
..
} if options_lift.instance == options_lower.instance => {
let index = *self
.runtime_always_trap_interner
.entry(canonical_abi)
.or_insert_with(|| {
let index =
RuntimeAlwaysTrapIndex::from_u32(self.result.num_always_trap);
self.result.num_always_trap += 1;
self.result.initializers.push(GlobalInitializer::AlwaysTrap(
AlwaysTrap {
canonical_abi,
index,
},
));
index
});
CoreDef::AlwaysTrap(index)
}
// Lowering a lifted function where the destination
// component is different than the source component means
// that a "fused adapter" was just identified.
//
// This is the location where, when this is actually
// implemented, we'll record metadata about this fused
// adapter to get compiled later during the compilation
// process. The fused adapter here will be generated by
// cranelift and will perfom argument validation when
// called, copy the arguments from `options_lower` to
// `options_lift` and then call the `func` specified for the
// lifted options.
//
// When the `func` returns the canonical adapter will verify
// the return values, copy them from `options_lift` to
// `options_lower`, and then return.
ComponentFuncDef::Lifted {
ty,
func,
options: options_lift,
} => {
// These are the various compilation options for lifting
// and lowering.
drop(ty); // component-model function type
drop(func); // original core wasm function that was lifted
drop(options_lift); // options during `canon lift`
drop(options_lower); // options during `canon lower`
drop(canonical_abi); // type signature of created core wasm function
unimplemented!("lowering a lifted function")
}
}
};
frame.funcs.push(func);
}
// Lifting a core wasm function is relatively easy for now in that
@@ -656,7 +710,7 @@ impl<'a> Inliner<'a> {
frame.tables.push(
match self.core_def_of_module_instance_export(frame, *instance, *name) {
CoreDef::Export(e) => e,
CoreDef::Lowered(_) => unreachable!(),
CoreDef::Lowered(_) | CoreDef::AlwaysTrap(_) => unreachable!(),
},
);
}
@@ -665,7 +719,7 @@ impl<'a> Inliner<'a> {
frame.globals.push(
match self.core_def_of_module_instance_export(frame, *instance, *name) {
CoreDef::Export(e) => e,
CoreDef::Lowered(_) => unreachable!(),
CoreDef::Lowered(_) | CoreDef::AlwaysTrap(_) => unreachable!(),
},
);
}
@@ -674,7 +728,7 @@ impl<'a> Inliner<'a> {
frame.memories.push(
match self.core_def_of_module_instance_export(frame, *instance, *name) {
CoreDef::Export(e) => e,
CoreDef::Lowered(_) => unreachable!(),
CoreDef::Lowered(_) | CoreDef::AlwaysTrap(_) => unreachable!(),
},
);
}

3
crates/environ/src/component/types.rs
View File

@@ -145,6 +145,9 @@ indices! {
/// component model.
pub struct LoweredIndex(u32);
/// Same as `LoweredIndex` but for the `CoreDef::AlwaysTrap` variant.
pub struct RuntimeAlwaysTrapIndex(u32);
/// Index representing a linear memory extracted from a wasm instance
/// which is stored in a `VMComponentContext`. This is used to deduplicate
/// references to the same linear memory where it's only stored once in a

26
crates/environ/src/component/vmcomponent_offsets.rs
View File

@@ -5,6 +5,7 @@
// store: *mut dyn Store,
// flags: [VMComponentFlags; component.num_runtime_component_instances],
// lowering_anyfuncs: [VMCallerCheckedAnyfunc; component.num_lowerings],
// always_trap_anyfuncs: [VMCallerCheckedAnyfunc; component.num_always_trap],
// lowerings: [VMLowering; component.num_lowerings],
// memories: [*mut VMMemoryDefinition; component.num_memories],
// reallocs: [*mut VMCallerCheckedAnyfunc; component.num_reallocs],
@@ -12,8 +13,8 @@
// }
use crate::component::{
Component, LoweredIndex, RuntimeComponentInstanceIndex, RuntimeMemoryIndex,
RuntimePostReturnIndex, RuntimeReallocIndex,
Component, LoweredIndex, RuntimeAlwaysTrapIndex, RuntimeComponentInstanceIndex,
RuntimeMemoryIndex, RuntimePostReturnIndex, RuntimeReallocIndex,
};
use crate::PtrSize;
@@ -52,12 +53,16 @@ pub struct VMComponentOffsets<P> {
/// Number of component instances internally in the component (always at
/// least 1).
pub num_runtime_component_instances: u32,
/// Number of "always trap" functions which have their
/// `VMCallerCheckedAnyfunc` stored inline in the `VMComponentContext`.
pub num_always_trap: u32,
// precalculated offsets of various member fields
magic: u32,
store: u32,
flags: u32,
lowering_anyfuncs: u32,
always_trap_anyfuncs: u32,
lowerings: u32,
memories: u32,
reallocs: u32,
@@ -85,10 +90,12 @@ impl<P: PtrSize> VMComponentOffsets<P> {
.num_runtime_component_instances
.try_into()
.unwrap(),
num_always_trap: component.num_always_trap,
magic: 0,
store: 0,
flags: 0,
lowering_anyfuncs: 0,
always_trap_anyfuncs: 0,
lowerings: 0,
memories: 0,
reallocs: 0,
@@ -127,6 +134,7 @@ impl<P: PtrSize> VMComponentOffsets<P> {
size(flags) = cmul(ret.num_runtime_component_instances, ret.size_of_vmcomponent_flags()),
align(u32::from(ret.ptr.size())),
size(lowering_anyfuncs) = cmul(ret.num_lowerings, ret.ptr.size_of_vmcaller_checked_anyfunc()),
size(always_trap_anyfuncs) = cmul(ret.num_always_trap, ret.ptr.size_of_vmcaller_checked_anyfunc()),
size(lowerings) = cmul(ret.num_lowerings, ret.ptr.size() * 2),
size(memories) = cmul(ret.num_runtime_memories, ret.ptr.size()),
size(reallocs) = cmul(ret.num_runtime_reallocs, ret.ptr.size()),
@@ -188,6 +196,20 @@ impl<P: PtrSize> VMComponentOffsets<P> {
+ index.as_u32() * u32::from(self.ptr.size_of_vmcaller_checked_anyfunc())
}
/// The offset of the `always_trap_anyfuncs` field.
#[inline]
pub fn always_trap_anyfuncs(&self) -> u32 {
self.always_trap_anyfuncs
}
/// The offset of `VMCallerCheckedAnyfunc` for the `index` specified.
#[inline]
pub fn always_trap_anyfunc(&self, index: RuntimeAlwaysTrapIndex) -> u32 {
assert!(index.as_u32() < self.num_always_trap);
self.always_trap_anyfuncs()
+ index.as_u32() * u32::from(self.ptr.size_of_vmcaller_checked_anyfunc())
}
/// The offset of the `lowerings` field.
#[inline]
pub fn lowerings(&self) -> u32 {

5
crates/environ/src/trap_encoding.rs
View File

@@ -98,6 +98,10 @@ pub enum TrapCode {
/// Execution has potentially run too long and may be interrupted.
/// This trap is resumable.
Interrupt,
/// Used for the component model when functions are lifted/lowered in a way
/// that generates a function that always traps.
AlwaysTrapAdapter,
// if adding a variant here be sure to update the `check!` macro below
}
@@ -205,6 +209,7 @@ pub fn lookup_trap_code(section: &[u8], offset: usize) -> Option<TrapCode> {
BadConversionToInteger
UnreachableCodeReached
Interrupt
AlwaysTrapAdapter
}
if cfg!(debug_assertions) {