T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Implement AOT compilation for components (#5160)

Browse Source 
* Pull `Module` out of `ModuleTextBuilder`

This commit is the first in what will likely be a number towards
preparing for serializing a compiled component to bytes, a precompiled
artifact. To that end my rough plan is to merge all of the compiled
artifacts for a component into one large object file instead of having
lots of separate object files and lots of separate mmaps to manage. To
that end I plan on eventually using `ModuleTextBuilder` to build one
large text section for all core wasm modules and trampolines, meaning
that `ModuleTextBuilder` is no longer specific to one module. I've
extracted out functionality such as function name calculation as well as
relocation resolving (now a closure passed in) in preparation for this.

For now this just keeps tests passing, and the trajectory for this
should become more clear over the following commits.

* Remove component-specific object emission

This commit removes the `ComponentCompiler::emit_obj` function in favor
of `Compiler::emit_obj`, now renamed `append_code`. This involved
significantly refactoring code emission to take a flat list of functions
into `append_code` and the caller is responsible for weaving together
various "families" of functions and un-weaving them afterwards.

* Consolidate ELF parsing in `CodeMemory`

This commit moves the ELF file parsing and section iteration from
`CompiledModule` into `CodeMemory` so one location keeps track of
section ranges and such. This is in preparation for sharing much of this
code with components which needs all the same sections to get tracked
but won't be using `CompiledModule`. A small side benefit from this is
that the section parsing done in `CodeMemory` and `CompiledModule` is no
longer duplicated.

* Remove separately tracked traps in components

Previously components would generate an "always trapping" function
and the metadata around which pc was allowed to trap was handled
manually for components. With recent refactorings the Wasmtime-standard
trap section in object files is now being generated for components as
well which means that can be reused instead of custom-tracking this
metadata. This commit removes the manual tracking for the `always_trap`
functions and plumbs the necessary bits around to make components look
more like modules.

* Remove a now-unnecessary `Arc` in `Module`

Not expected to have any measurable impact on performance, but
complexity-wise this should make it a bit easier to understand the
internals since there's no longer any need to store this somewhere else
than its owner's location.

* Merge compilation artifacts of components

This commit is a large refactoring of the component compilation process
to produce a single artifact instead of multiple binary artifacts. The
core wasm compilation process is refactored as well to share as much
code as necessary with the component compilation process.

This method of representing a compiled component necessitated a few
medium-sized changes internally within Wasmtime:

* A new data structure was created, `CodeObject`, which represents
  metadata about a single compiled artifact. This is then stored as an
  `Arc` within a component and a module. For `Module` this is always
  uniquely owned and represents a shuffling around of data from one
  owner to another. For a `Component`, however, this is shared amongst
  all loaded modules and the top-level component.

* The "module registry" which is used for symbolicating backtraces and
  for trap information has been updated to account for a single region
  of loaded code holding possibly multiple modules. This involved adding
  a second-level `BTreeMap` for now. This will likely slow down
  instantiation slightly but if it poses an issue in the future this
  should be able to be represented with a more clever data structure.

This commit additionally solves a number of longstanding issues with
components such as compiling only one host-to-wasm trampoline per
signature instead of possibly once-per-module. Additionally the
`SignatureCollection` registration now happens once-per-component
instead of once-per-module-within-a-component.

* Fix compile errors from prior commits

* Support AOT-compiling components

This commit adds support for AOT-compiled components in the same manner
as `Module`, specifically adding:

* `Engine::precompile_component`
* `Component::serialize`
* `Component::deserialize`
* `Component::deserialize_file`

Internally the support for components looks quite similar to `Module`.
All the prior commits to this made adding the support here
(unsurprisingly) easy. Components are represented as a single object
file as are modules, and the functions for each module are all piled
into the same object file next to each other (as are areas such as data
sections). Support was also added here to quickly differentiate compiled
components vs compiled modules via the `e_flags` field in the ELF
header.

* Prevent serializing exported modules on components

The current representation of a module within a component means that the
implementation of `Module::serialize` will not work if the module is
exported from a component. The reason for this is that `serialize`
doesn't actually do anything and simply returns the underlying mmap as a
list of bytes. The mmap, however, has `.wasmtime.info` describing
component metadata as opposed to this module's metadata. While rewriting
this section could be implemented it's not so easy to do so and is
otherwise seen as not super important of a feature right now anyway.

* Fix windows build

* Fix an unused function warning

* Update crates/environ/src/compilation.rs

Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>

Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
This commit is contained in:
Alex Crichton
2022-11-02 10:26:26 -05:00
committed by GitHub
parent 033758daaf
commit cd53bed898
45 changed files with 1991 additions and 1763 deletions
Download Patch File Download Diff File Expand all files Collapse all files

312
crates/jit/src/code_memory.rs
View File

@@ -1,16 +1,16 @@
//! Memory management for executable code.
use crate::subslice_range;
use crate::unwind::UnwindRegistration;
use anyhow::{anyhow, bail, Context, Result};
use object::read::{File, Object, ObjectSection};
use std::ffi::c_void;
use std::mem;
use std::mem::ManuallyDrop;
use std::ops::Range;
use wasmtime_environ::obj;
use wasmtime_environ::FunctionLoc;
use wasmtime_jit_icache_coherence as icache_coherence;
use wasmtime_runtime::MmapVec;
/// Name of the section in ELF files indicating that branch protection was
/// enabled for the compiled code.
pub const ELF_WASM_BTI: &str = ".wasmtime.bti";
use wasmtime_runtime::{MmapVec, VMTrampoline};
/// Management of executable memory within a `MmapVec`
///
@@ -22,6 +22,20 @@ pub struct CodeMemory {
mmap: ManuallyDrop<MmapVec>,
unwind_registration: ManuallyDrop<Option<UnwindRegistration>>,
published: bool,
enable_branch_protection: bool,
// Ranges within `self.mmap` of where the particular sections lie.
text: Range<usize>,
unwind: Range<usize>,
trap_data: Range<usize>,
wasm_data: Range<usize>,
address_map_data: Range<usize>,
func_name_data: Range<usize>,
info_data: Range<usize>,
/// Map of dwarf sections indexed by `gimli::SectionId` which points to the
/// range within `code_memory`'s mmap as to the contents of the section.
dwarf_sections: Vec<Range<usize>>,
}
impl Drop for CodeMemory {
@@ -39,32 +53,115 @@ fn _assert() {
_assert_send_sync::<CodeMemory>();
}
/// Result of publishing a `CodeMemory`, containing references to the parsed
/// internals.
pub struct Publish<'a> {
/// The parsed ELF image that resides within the original `MmapVec`.
pub obj: File<'a>,
/// Reference to the entire `MmapVec` and its contents.
pub mmap: &'a MmapVec,
/// Reference to just the text section of the object file, a subslice of
/// `mmap`.
pub text: &'a [u8],
}
impl CodeMemory {
/// Creates a new `CodeMemory` by taking ownership of the provided
/// `MmapVec`.
///
/// The returned `CodeMemory` manages the internal `MmapVec` and the
/// `publish` method is used to actually make the memory executable.
pub fn new(mmap: MmapVec) -> Self {
Self {
pub fn new(mmap: MmapVec) -> Result<Self> {
use gimli::SectionId::*;
let obj = File::parse(&mmap[..])
.with_context(|| "failed to parse internal compilation artifact")?;
let mut text = 0..0;
let mut unwind = 0..0;
let mut enable_branch_protection = None;
let mut trap_data = 0..0;
let mut wasm_data = 0..0;
let mut address_map_data = 0..0;
let mut func_name_data = 0..0;
let mut info_data = 0..0;
let mut dwarf_sections = Vec::new();
for section in obj.sections() {
let data = section.data()?;
let name = section.name()?;
let range = subslice_range(data, &mmap);
// Double-check that sections are all aligned properly.
if section.align() != 0 && data.len() != 0 {
if (data.as_ptr() as u64 - mmap.as_ptr() as u64) % section.align() != 0 {
bail!(
"section `{}` isn't aligned to {:#x}",
section.name().unwrap_or("ERROR"),
section.align()
);
}
}
let mut gimli = |id: gimli::SectionId| {
let idx = id as usize;
if dwarf_sections.len() <= idx {
dwarf_sections.resize(idx + 1, 0..0);
}
dwarf_sections[idx] = range.clone();
};
match name {
obj::ELF_WASM_BTI => match data.len() {
1 => enable_branch_protection = Some(data[0] != 0),
_ => bail!("invalid `{name}` section"),
},
".text" => {
text = range;
// Double-check there are no relocations in the text section. At
// this time relocations are not expected at all from loaded code
// since everything should be resolved at compile time. Handling
// must be added here, though, if relocations pop up.
assert!(section.relocations().count() == 0);
}
UnwindRegistration::SECTION_NAME => unwind = range,
obj::ELF_WASM_DATA => wasm_data = range,
obj::ELF_WASMTIME_ADDRMAP => address_map_data = range,
obj::ELF_WASMTIME_TRAPS => trap_data = range,
obj::ELF_NAME_DATA => func_name_data = range,
obj::ELF_WASMTIME_INFO => info_data = range,
// Register dwarf sections into the `dwarf_sections`
// array which is indexed by `gimli::SectionId`
".debug_abbrev.wasm" => gimli(DebugAbbrev),
".debug_addr.wasm" => gimli(DebugAddr),
".debug_aranges.wasm" => gimli(DebugAranges),
".debug_frame.wasm" => gimli(DebugFrame),
".eh_frame.wasm" => gimli(EhFrame),
".eh_frame_hdr.wasm" => gimli(EhFrameHdr),
".debug_info.wasm" => gimli(DebugInfo),
".debug_line.wasm" => gimli(DebugLine),
".debug_line_str.wasm" => gimli(DebugLineStr),
".debug_loc.wasm" => gimli(DebugLoc),
".debug_loc_lists.wasm" => gimli(DebugLocLists),
".debug_macinfo.wasm" => gimli(DebugMacinfo),
".debug_macro.wasm" => gimli(DebugMacro),
".debug_pub_names.wasm" => gimli(DebugPubNames),
".debug_pub_types.wasm" => gimli(DebugPubTypes),
".debug_ranges.wasm" => gimli(DebugRanges),
".debug_rng_lists.wasm" => gimli(DebugRngLists),
".debug_str.wasm" => gimli(DebugStr),
".debug_str_offsets.wasm" => gimli(DebugStrOffsets),
".debug_types.wasm" => gimli(DebugTypes),
".debug_cu_index.wasm" => gimli(DebugCuIndex),
".debug_tu_index.wasm" => gimli(DebugTuIndex),
_ => log::debug!("ignoring section {name}"),
}
}
Ok(Self {
mmap: ManuallyDrop::new(mmap),
unwind_registration: ManuallyDrop::new(None),
published: false,
}
enable_branch_protection: enable_branch_protection
.ok_or_else(|| anyhow!("missing `{}` section", obj::ELF_WASM_BTI))?,
text,
unwind,
trap_data,
address_map_data,
func_name_data,
dwarf_sections,
info_data,
wasm_data,
})
}
/// Returns a reference to the underlying `MmapVec` this memory owns.
@@ -72,6 +169,67 @@ impl CodeMemory {
&self.mmap
}
/// Returns the contents of the text section of the ELF executable this
/// represents.
pub fn text(&self) -> &[u8] {
&self.mmap[self.text.clone()]
}
/// Returns the data in the corresponding dwarf section, or an empty slice
/// if the section wasn't present.
pub fn dwarf_section(&self, section: gimli::SectionId) -> &[u8] {
let range = self
.dwarf_sections
.get(section as usize)
.cloned()
.unwrap_or(0..0);
&self.mmap[range]
}
/// Returns the data in the `ELF_NAME_DATA` section.
pub fn func_name_data(&self) -> &[u8] {
&self.mmap[self.func_name_data.clone()]
}
/// Returns the concatenated list of all data associated with this wasm
/// module.
///
/// This is used for initialization of memories and all data ranges stored
/// in a `Module` are relative to the slice returned here.
pub fn wasm_data(&self) -> &[u8] {
&self.mmap[self.wasm_data.clone()]
}
/// Returns the encoded address map section used to pass to
/// `wasmtime_environ::lookup_file_pos`.
pub fn address_map_data(&self) -> &[u8] {
&self.mmap[self.address_map_data.clone()]
}
/// Returns the contents of the `ELF_WASMTIME_INFO` section, or an empty
/// slice if it wasn't found.
pub fn wasmtime_info(&self) -> &[u8] {
&self.mmap[self.info_data.clone()]
}
/// Returns the contents of the `ELF_WASMTIME_TRAPS` section, or an empty
/// slice if it wasn't found.
pub fn trap_data(&self) -> &[u8] {
&self.mmap[self.trap_data.clone()]
}
/// Returns a `VMTrampoline` function pointer for the given function in the
/// text section.
///
/// # Unsafety
///
/// This function is unsafe as there's no guarantee that the returned
/// function pointer is valid.
pub unsafe fn vmtrampoline(&self, loc: FunctionLoc) -> VMTrampoline {
let ptr = self.text()[loc.start as usize..][..loc.length as usize].as_ptr();
mem::transmute::<*const u8, VMTrampoline>(ptr)
}
/// Publishes the internal ELF image to be ready for execution.
///
/// This method can only be called once and will panic if called twice. This
@@ -82,99 +240,37 @@ impl CodeMemory {
/// * Register unwinding information with the OS
///
/// After this function executes all JIT code should be ready to execute.
/// The various parsed results of the internals of the `MmapVec` are
/// returned through the `Publish` structure.
pub fn publish(&mut self) -> Result<Publish<'_>> {
pub fn publish(&mut self) -> Result<()> {
assert!(!self.published);
self.published = true;
let mut ret = Publish {
obj: File::parse(&self.mmap[..])
.with_context(|| "failed to parse internal compilation artifact")?,
mmap: &self.mmap,
text: &[],
};
let mmap_ptr = self.mmap.as_ptr() as u64;
// Sanity-check that all sections are aligned correctly and
// additionally probe for a few sections that we're interested in.
let mut enable_branch_protection = None;
let mut text = None;
for section in ret.obj.sections() {
let data = match section.data() {
Ok(data) => data,
Err(_) => continue,
};
if section.align() == 0 || data.len() == 0 {
continue;
}
if (data.as_ptr() as u64 - mmap_ptr) % section.align() != 0 {
bail!(
"section `{}` isn't aligned to {:#x}",
section.name().unwrap_or("ERROR"),
section.align()
);
}
match section.name().unwrap_or("") {
ELF_WASM_BTI => match data.len() {
1 => enable_branch_protection = Some(data[0] != 0),
_ => bail!("invalid `{ELF_WASM_BTI}` section"),
},
".text" => {
ret.text = data;
text = Some(section);
}
_ => {}
}
if self.text().is_empty() {
return Ok(());
}
let enable_branch_protection =
enable_branch_protection.ok_or_else(|| anyhow!("missing `{ELF_WASM_BTI}` section"))?;
let text = match text {
Some(text) => text,
None => return Ok(ret),
};
// The unsafety here comes from a few things:
//
// * First in `apply_reloc` we're walking around the `File` that the
// `object` crate has to get a mutable view into the text section.
// Currently the `object` crate doesn't support easily parsing a file
// and updating small bits and pieces of it, so we work around it for
// now. ELF's file format should guarantee that `text_mut` doesn't
// collide with any memory accessed by `text.relocations()`.
// * We're actually updating some page protections to executable memory.
//
// * Second we're actually updating some page protections to executable
// memory.
//
// * Finally we're registering unwinding information which relies on the
// * We're registering unwinding information which relies on the
// correctness of the information in the first place. This applies to
// both the actual unwinding tables as well as the validity of the
// pointers we pass in itself.
unsafe {
let text_mut =
std::slice::from_raw_parts_mut(ret.text.as_ptr() as *mut u8, ret.text.len());
let text_offset = ret.text.as_ptr() as usize - ret.mmap.as_ptr() as usize;
let text_range = text_offset..text_offset + text_mut.len();
// Double-check there are no relocations in the text section. At
// this time relocations are not expected at all from loaded code
// since everything should be resolved at compile time. Handling
// must be added here, though, if relocations pop up.
assert!(text.relocations().count() == 0);
let text = self.text();
// Clear the newly allocated code from cache if the processor requires it
//
// Do this before marking the memory as R+X, technically we should be able to do it after
// but there are some CPU's that have had errata about doing this with read only memory.
icache_coherence::clear_cache(ret.text.as_ptr() as *const c_void, ret.text.len())
icache_coherence::clear_cache(text.as_ptr().cast(), text.len())
.expect("Failed cache clear");
// Switch the executable portion from read/write to
// read/execute, notably not using read/write/execute to prevent
// modifications.
self.mmap
.make_executable(text_range.clone(), enable_branch_protection)
.make_executable(self.text.clone(), self.enable_branch_protection)
.expect("unable to make memory executable");
// Flush any in-flight instructions from the pipeline
@@ -184,26 +280,22 @@ impl CodeMemory {
// `UnwindRegistration` implementation to inform the general
// runtime that there's unwinding information available for all
// our just-published JIT functions.
*self.unwind_registration = register_unwind_info(&ret.obj, ret.text)?;
self.register_unwind_info()?;
}
Ok(ret)
Ok(())
}
}
unsafe fn register_unwind_info(obj: &File, text: &[u8]) -> Result<Option<UnwindRegistration>> {
let unwind_info = match obj
.section_by_name(UnwindRegistration::section_name())
.and_then(|s| s.data().ok())
{
Some(info) => info,
None => return Ok(None),
};
if unwind_info.len() == 0 {
return Ok(None);
unsafe fn register_unwind_info(&mut self) -> Result<()> {
if self.unwind.len() == 0 {
return Ok(());
}
let text = self.text();
let unwind_info = &self.mmap[self.unwind.clone()];
let registration =
UnwindRegistration::new(text.as_ptr(), unwind_info.as_ptr(), unwind_info.len())
.context("failed to create unwind info registration")?;
*self.unwind_registration = Some(registration);
Ok(())
}
Ok(Some(
UnwindRegistration::new(text.as_ptr(), unwind_info.as_ptr(), unwind_info.len())
.context("failed to create unwind info registration")?,
))
}

720
crates/jit/src/instantiate.rs
View File

@@ -6,66 +6,25 @@
use crate::code_memory::CodeMemory;
use crate::debug::create_gdbjit_image;
use crate::ProfilingAgent;
use anyhow::{anyhow, bail, Context, Error, Result};
use object::write::{Object, StandardSegment, WritableBuffer};
use object::{File, Object as _, ObjectSection, SectionKind};
use anyhow::{bail, Context, Error, Result};
use object::write::{Object, SectionId, StandardSegment, WritableBuffer};
use object::SectionKind;
use serde::{Deserialize, Serialize};
use std::convert::TryFrom;
use std::ops::Range;
use std::str;
use std::sync::Arc;
use thiserror::Error;
use wasmtime_environ::obj;
use wasmtime_environ::{
CompileError, DefinedFuncIndex, FuncIndex, FunctionInfo, Module, ModuleTranslation, PrimaryMap,
SignatureIndex, StackMapInformation, Trampoline, Tunables, ELF_WASMTIME_ADDRMAP,
ELF_WASMTIME_TRAPS,
CompileError, DefinedFuncIndex, FuncIndex, FunctionLoc, MemoryInitialization, Module,
ModuleTranslation, PrimaryMap, SignatureIndex, StackMapInformation, Tunables, WasmFunctionInfo,
};
use wasmtime_runtime::{
CompiledModuleId, CompiledModuleIdAllocator, GdbJitImageRegistration, InstantiationError,
MmapVec, VMFunctionBody, VMTrampoline,
};
/// This is the name of the section in the final ELF image which contains
/// concatenated data segments from the original wasm module.
///
/// This section is simply a list of bytes and ranges into this section are
/// stored within a `Module` for each data segment. Memory initialization and
/// passive segment management all index data directly located in this section.
///
/// Note that this implementation does not afford any method of leveraging the
/// `data.drop` instruction to actually release the data back to the OS. The
/// data section is simply always present in the ELF image. If we wanted to
/// release the data it's probably best to figure out what the best
/// implementation is for it at the time given a particular set of constraints.
const ELF_WASM_DATA: &'static str = ".rodata.wasm";
/// This is the name of the section in the final ELF image which contains a
/// `bincode`-encoded `CompiledModuleInfo`.
///
/// This section is optionally decoded in `CompiledModule::from_artifacts`
/// depending on whether or not a `CompiledModuleInfo` is already available. In
/// cases like `Module::new` where compilation directly leads into consumption,
/// it's available. In cases like `Module::deserialize` this section must be
/// decoded to get all the relevant information.
const ELF_WASMTIME_INFO: &'static str = ".wasmtime.info";
/// This is the name of the section in the final ELF image which contains a
/// concatenated list of all function names.
///
/// This section is optionally included in the final artifact depending on
/// whether the wasm module has any name data at all (or in the future if we add
/// an option to not preserve name data). This section is a concatenated list of
/// strings where `CompiledModuleInfo::func_names` stores offsets/lengths into
/// this section.
///
/// Note that the goal of this section is to avoid having to decode names at
/// module-load time if we can. Names are typically only used for debugging or
/// things like backtraces so there's no need to eagerly load all of them. By
/// storing the data in a separate section the hope is that the data, which is
/// sometimes quite large (3MB seen for spidermonkey-compiled-to-wasm), can be
/// paged in lazily from an mmap and is never paged in if we never reference it.
const ELF_NAME_DATA: &'static str = ".name.wasm";
/// An error condition while setting up a wasm instance, be it validation,
/// compilation, or instantiation.
#[derive(Error, Debug)]
@@ -98,14 +57,14 @@ pub struct CompiledModuleInfo {
module: Module,
/// Metadata about each compiled function.
funcs: PrimaryMap<DefinedFuncIndex, FunctionInfo>,
funcs: PrimaryMap<DefinedFuncIndex, (WasmFunctionInfo, FunctionLoc)>,
/// Sorted list, by function index, of names we have for this module.
func_names: Vec<FunctionName>,
/// The trampolines compiled into the text section and their start/length
/// relative to the start of the text section.
trampolines: Vec<Trampoline>,
pub trampolines: Vec<(SignatureIndex, FunctionLoc)>,
/// General compilation metadata.
meta: Metadata,
@@ -138,365 +97,345 @@ struct Metadata {
has_wasm_debuginfo: bool,
}
/// Finishes compilation of the `translation` specified, producing the final
/// compilation artifact and auxiliary information.
/// Helper structure to create an ELF file as a compilation artifact.
///
/// This function will consume the final results of compiling a wasm module
/// and finish the ELF image in-progress as part of `obj` by appending any
/// compiler-agnostic sections.
///
/// The auxiliary `CompiledModuleInfo` structure returned here has also been
/// serialized into the object returned, but if the caller will quickly
/// turn-around and invoke `CompiledModule::from_artifacts` after this then the
/// information can be passed to that method to avoid extra deserialization.
/// This is done to avoid a serialize-then-deserialize for API calls like
/// `Module::new` where the compiled module is immediately going to be used.
///
/// The `MmapVec` returned here contains the compiled image and resides in
/// mmap'd memory for easily switching permissions to executable afterwards.
pub fn finish_compile(
translation: ModuleTranslation<'_>,
mut obj: Object,
funcs: PrimaryMap<DefinedFuncIndex, FunctionInfo>,
trampolines: Vec<Trampoline>,
tunables: &Tunables,
) -> Result<(MmapVec, CompiledModuleInfo)> {
let ModuleTranslation {
mut module,
debuginfo,
has_unparsed_debuginfo,
data,
data_align,
passive_data,
..
} = translation;
/// This structure exposes the process which Wasmtime will encode a core wasm
/// module into an ELF file, notably managing data sections and all that good
/// business going into the final file.
pub struct ObjectBuilder<'a> {
/// The `object`-crate-defined ELF file write we're using.
obj: Object<'a>,
// Place all data from the wasm module into a section which will the
// source of the data later at runtime.
let data_id = obj.add_section(
obj.segment_name(StandardSegment::Data).to_vec(),
ELF_WASM_DATA.as_bytes().to_vec(),
SectionKind::ReadOnlyData,
);
let mut total_data_len = 0;
for (i, data) in data.iter().enumerate() {
// The first data segment has its alignment specified as the alignment
// for the entire section, but everything afterwards is adjacent so it
// has alignment of 1.
let align = if i == 0 { data_align.unwrap_or(1) } else { 1 };
obj.append_section_data(data_id, data, align);
total_data_len += data.len();
}
for data in passive_data.iter() {
obj.append_section_data(data_id, data, 1);
}
/// General compilation configuration.
tunables: &'a Tunables,
// If any names are present in the module then the `ELF_NAME_DATA` section
// is create and appended.
let mut func_names = Vec::new();
if debuginfo.name_section.func_names.len() > 0 {
let name_id = obj.add_section(
/// The section identifier for "rodata" which is where wasm data segments
/// will go.
data: SectionId,
/// The section identifier for function name information, or otherwise where
/// the `name` custom section of wasm is copied into.
///
/// This is optional and lazily created on demand.
names: Option<SectionId>,
}
impl<'a> ObjectBuilder<'a> {
/// Creates a new builder for the `obj` specified.
pub fn new(mut obj: Object<'a>, tunables: &'a Tunables) -> ObjectBuilder<'a> {
let data = obj.add_section(
obj.segment_name(StandardSegment::Data).to_vec(),
ELF_NAME_DATA.as_bytes().to_vec(),
obj::ELF_WASM_DATA.as_bytes().to_vec(),
SectionKind::ReadOnlyData,
);
let mut sorted_names = debuginfo.name_section.func_names.iter().collect::<Vec<_>>();
sorted_names.sort_by_key(|(idx, _name)| *idx);
for (idx, name) in sorted_names {
let offset = obj.append_section_data(name_id, name.as_bytes(), 1);
let offset = match u32::try_from(offset) {
Ok(offset) => offset,
Err(_) => bail!("name section too large (> 4gb)"),
};
let len = u32::try_from(name.len()).unwrap();
func_names.push(FunctionName {
idx: *idx,
offset,
len,
});
ObjectBuilder {
obj,
tunables,
data,
names: None,
}
}
// Update passive data offsets since they're all located after the other
// data in the module.
for (_, range) in module.passive_data_map.iter_mut() {
range.start = range.start.checked_add(total_data_len as u32).unwrap();
range.end = range.end.checked_add(total_data_len as u32).unwrap();
}
// Insert the wasm raw wasm-based debuginfo into the output, if
// requested. Note that this is distinct from the native debuginfo
// possibly generated by the native compiler, hence these sections
// getting wasm-specific names.
if tunables.parse_wasm_debuginfo {
push_debug(&mut obj, &debuginfo.dwarf.debug_abbrev);
push_debug(&mut obj, &debuginfo.dwarf.debug_addr);
push_debug(&mut obj, &debuginfo.dwarf.debug_aranges);
push_debug(&mut obj, &debuginfo.dwarf.debug_info);
push_debug(&mut obj, &debuginfo.dwarf.debug_line);
push_debug(&mut obj, &debuginfo.dwarf.debug_line_str);
push_debug(&mut obj, &debuginfo.dwarf.debug_str);
push_debug(&mut obj, &debuginfo.dwarf.debug_str_offsets);
push_debug(&mut obj, &debuginfo.debug_ranges);
push_debug(&mut obj, &debuginfo.debug_rnglists);
}
// Encode a `CompiledModuleInfo` structure into the `ELF_WASMTIME_INFO`
// section of this image. This is not necessary when the returned module
// is never serialized to disk, which is also why we return a copy of
// the `CompiledModuleInfo` structure to the caller in case they don't
// want to deserialize this value immediately afterwards from the
// section. Otherwise, though, this is necessary to reify a `Module` on
// the other side from disk-serialized artifacts in
// `Module::deserialize` (a Wasmtime API).
let info_id = obj.add_section(
obj.segment_name(StandardSegment::Data).to_vec(),
ELF_WASMTIME_INFO.as_bytes().to_vec(),
SectionKind::ReadOnlyData,
);
let mut bytes = Vec::new();
let info = CompiledModuleInfo {
module,
funcs,
trampolines,
func_names,
meta: Metadata {
native_debug_info_present: tunables.generate_native_debuginfo,
/// Completes compilation of the `translation` specified, inserting
/// everything necessary into the `Object` being built.
///
/// This function will consume the final results of compiling a wasm module
/// and finish the ELF image in-progress as part of `self.obj` by appending
/// any compiler-agnostic sections.
///
/// The auxiliary `CompiledModuleInfo` structure returned here has also been
/// serialized into the object returned, but if the caller will quickly
/// turn-around and invoke `CompiledModule::from_artifacts` after this then
/// the information can be passed to that method to avoid extra
/// deserialization. This is done to avoid a serialize-then-deserialize for
/// API calls like `Module::new` where the compiled module is immediately
/// going to be used.
///
/// The various arguments here are:
///
/// * `translation` - the core wasm translation that's being completed.
///
/// * `funcs` - compilation metadata about functions within the translation
/// as well as where the functions are located in the text section.
///
/// * `trampolines` - list of all trampolines necessary for this module
/// and where they're located in the text section.
///
/// Returns the `CompiledModuleInfo` corresopnding to this core wasm module
/// as a result of this append operation. This is then serialized into the
/// final artifact by the caller.
pub fn append(
&mut self,
translation: ModuleTranslation<'_>,
funcs: PrimaryMap<DefinedFuncIndex, (WasmFunctionInfo, FunctionLoc)>,
trampolines: Vec<(SignatureIndex, FunctionLoc)>,
) -> Result<CompiledModuleInfo> {
let ModuleTranslation {
mut module,
debuginfo,
has_unparsed_debuginfo,
code_section_offset: debuginfo.wasm_file.code_section_offset,
has_wasm_debuginfo: tunables.parse_wasm_debuginfo,
},
};
bincode::serialize_into(&mut bytes, &info)?;
obj.append_section_data(info_id, &bytes, 1);
data,
data_align,
passive_data,
..
} = translation;
return Ok((mmap_vec_from_obj(obj)?, info));
// Place all data from the wasm module into a section which will the
// source of the data later at runtime. This additionally keeps track of
// the offset of
let mut total_data_len = 0;
let data_offset = self
.obj
.append_section_data(self.data, &[], data_align.unwrap_or(1));
for (i, data) in data.iter().enumerate() {
// The first data segment has its alignment specified as the alignment
// for the entire section, but everything afterwards is adjacent so it
// has alignment of 1.
let align = if i == 0 { data_align.unwrap_or(1) } else { 1 };
self.obj.append_section_data(self.data, data, align);
total_data_len += data.len();
}
for data in passive_data.iter() {
self.obj.append_section_data(self.data, data, 1);
}
fn push_debug<'a, T>(obj: &mut Object, section: &T)
// If any names are present in the module then the `ELF_NAME_DATA` section
// is create and appended.
let mut func_names = Vec::new();
if debuginfo.name_section.func_names.len() > 0 {
let name_id = *self.names.get_or_insert_with(|| {
self.obj.add_section(
self.obj.segment_name(StandardSegment::Data).to_vec(),
obj::ELF_NAME_DATA.as_bytes().to_vec(),
SectionKind::ReadOnlyData,
)
});
let mut sorted_names = debuginfo.name_section.func_names.iter().collect::<Vec<_>>();
sorted_names.sort_by_key(|(idx, _name)| *idx);
for (idx, name) in sorted_names {
let offset = self.obj.append_section_data(name_id, name.as_bytes(), 1);
let offset = match u32::try_from(offset) {
Ok(offset) => offset,
Err(_) => bail!("name section too large (> 4gb)"),
};
let len = u32::try_from(name.len()).unwrap();
func_names.push(FunctionName {
idx: *idx,
offset,
len,
});
}
}
// Data offsets in `MemoryInitialization` are offsets within the
// `translation.data` list concatenated which is now present in the data
// segment that's appended to the object. Increase the offsets by
// `self.data_size` to account for any previously added module.
let data_offset = u32::try_from(data_offset).unwrap();
match &mut module.memory_initialization {
MemoryInitialization::Segmented(list) => {
for segment in list {
segment.data.start = segment.data.start.checked_add(data_offset).unwrap();
segment.data.end = segment.data.end.checked_add(data_offset).unwrap();
}
}
MemoryInitialization::Static { map } => {
for (_, segment) in map {
if let Some(segment) = segment {
segment.data.start = segment.data.start.checked_add(data_offset).unwrap();
segment.data.end = segment.data.end.checked_add(data_offset).unwrap();
}
}
}
}
// Data offsets for passive data are relative to the start of
// `translation.passive_data` which was appended to the data segment
// of this object, after active data in `translation.data`. Update the
// offsets to account prior modules added in addition to active data.
let data_offset = data_offset + u32::try_from(total_data_len).unwrap();
for (_, range) in module.passive_data_map.iter_mut() {
range.start = range.start.checked_add(data_offset).unwrap();
range.end = range.end.checked_add(data_offset).unwrap();
}
// Insert the wasm raw wasm-based debuginfo into the output, if
// requested. Note that this is distinct from the native debuginfo
// possibly generated by the native compiler, hence these sections
// getting wasm-specific names.
if self.tunables.parse_wasm_debuginfo {
self.push_debug(&debuginfo.dwarf.debug_abbrev);
self.push_debug(&debuginfo.dwarf.debug_addr);
self.push_debug(&debuginfo.dwarf.debug_aranges);
self.push_debug(&debuginfo.dwarf.debug_info);
self.push_debug(&debuginfo.dwarf.debug_line);
self.push_debug(&debuginfo.dwarf.debug_line_str);
self.push_debug(&debuginfo.dwarf.debug_str);
self.push_debug(&debuginfo.dwarf.debug_str_offsets);
self.push_debug(&debuginfo.debug_ranges);
self.push_debug(&debuginfo.debug_rnglists);
}
Ok(CompiledModuleInfo {
module,
funcs,
trampolines,
func_names,
meta: Metadata {
native_debug_info_present: self.tunables.generate_native_debuginfo,
has_unparsed_debuginfo,
code_section_offset: debuginfo.wasm_file.code_section_offset,
has_wasm_debuginfo: self.tunables.parse_wasm_debuginfo,
},
})
}
fn push_debug<'b, T>(&mut self, section: &T)
where
T: gimli::Section<gimli::EndianSlice<'a, gimli::LittleEndian>>,
T: gimli::Section<gimli::EndianSlice<'b, gimli::LittleEndian>>,
{
let data = section.reader().slice();
if data.is_empty() {
return;
}
let section_id = obj.add_section(
obj.segment_name(StandardSegment::Debug).to_vec(),
let section_id = self.obj.add_section(
self.obj.segment_name(StandardSegment::Debug).to_vec(),
format!("{}.wasm", T::id().name()).into_bytes(),
SectionKind::Debug,
);
obj.append_section_data(section_id, data, 1);
self.obj.append_section_data(section_id, data, 1);
}
}
/// Creates a new `MmapVec` from serializing the specified `obj`.
///
/// The returned `MmapVec` will contain the serialized version of `obj` and
/// is sized appropriately to the exact size of the object serialized.
pub fn mmap_vec_from_obj(obj: Object) -> Result<MmapVec> {
let mut result = ObjectMmap::default();
return match obj.emit(&mut result) {
Ok(()) => {
assert!(result.mmap.is_some(), "no reserve");
let mmap = result.mmap.expect("reserve not called");
assert_eq!(mmap.len(), result.len);
Ok(mmap)
}
Err(e) => match result.err.take() {
Some(original) => Err(original.context(e)),
None => Err(e.into()),
},
};
/// Creates the `ELF_WASMTIME_INFO` section from the given serializable data
/// structure.
pub fn serialize_info<T>(&mut self, info: &T)
where
T: serde::Serialize,
{
let section = self.obj.add_section(
self.obj.segment_name(StandardSegment::Data).to_vec(),
obj::ELF_WASMTIME_INFO.as_bytes().to_vec(),
SectionKind::ReadOnlyData,
);
let data = bincode::serialize(info).unwrap();
self.obj.set_section_data(section, data, 1);
}
/// Helper struct to implement the `WritableBuffer` trait from the `object`
/// crate.
/// Creates a new `MmapVec` from `self.`
///
/// This enables writing an object directly into an mmap'd memory so it's
/// immediately usable for execution after compilation. This implementation
/// relies on a call to `reserve` happening once up front with all the needed
/// data, and the mmap internally does not attempt to grow afterwards.
#[derive(Default)]
struct ObjectMmap {
mmap: Option<MmapVec>,
len: usize,
err: Option<Error>,
}
impl WritableBuffer for ObjectMmap {
fn len(&self) -> usize {
self.len
}
fn reserve(&mut self, additional: usize) -> Result<(), ()> {
assert!(self.mmap.is_none(), "cannot reserve twice");
self.mmap = match MmapVec::with_capacity(additional) {
Ok(mmap) => Some(mmap),
Err(e) => {
self.err = Some(e);
return Err(());
}
};
Ok(())
}
fn resize(&mut self, new_len: usize) {
// Resizing always appends 0 bytes and since new mmaps start out as 0
// bytes we don't actually need to do anything as part of this other
// than update our own length.
if new_len <= self.len {
return;
/// The returned `MmapVec` will contain the serialized version of `self`
/// and is sized appropriately to the exact size of the object serialized.
pub fn finish(self) -> Result<MmapVec> {
let mut result = ObjectMmap::default();
return match self.obj.emit(&mut result) {
Ok(()) => {
assert!(result.mmap.is_some(), "no reserve");
let mmap = result.mmap.expect("reserve not called");
assert_eq!(mmap.len(), result.len);
Ok(mmap)
}
self.len = new_len;
Err(e) => match result.err.take() {
Some(original) => Err(original.context(e)),
None => Err(e.into()),
},
};
/// Helper struct to implement the `WritableBuffer` trait from the `object`
/// crate.
///
/// This enables writing an object directly into an mmap'd memory so it's
/// immediately usable for execution after compilation. This implementation
/// relies on a call to `reserve` happening once up front with all the needed
/// data, and the mmap internally does not attempt to grow afterwards.
#[derive(Default)]
struct ObjectMmap {
mmap: Option<MmapVec>,
len: usize,
err: Option<Error>,
}
fn write_bytes(&mut self, val: &[u8]) {
let mmap = self.mmap.as_mut().expect("write before reserve");
mmap[self.len..][..val.len()].copy_from_slice(val);
self.len += val.len();
impl WritableBuffer for ObjectMmap {
fn len(&self) -> usize {
self.len
}
fn reserve(&mut self, additional: usize) -> Result<(), ()> {
assert!(self.mmap.is_none(), "cannot reserve twice");
self.mmap = match MmapVec::with_capacity(additional) {
Ok(mmap) => Some(mmap),
Err(e) => {
self.err = Some(e);
return Err(());
}
};
Ok(())
}
fn resize(&mut self, new_len: usize) {
// Resizing always appends 0 bytes and since new mmaps start out as 0
// bytes we don't actually need to do anything as part of this other
// than update our own length.
if new_len <= self.len {
return;
}
self.len = new_len;
}
fn write_bytes(&mut self, val: &[u8]) {
let mmap = self.mmap.as_mut().expect("write before reserve");
mmap[self.len..][..val.len()].copy_from_slice(val);
self.len += val.len();
}
}
}
}
/// A compiled wasm module, ready to be instantiated.
pub struct CompiledModule {
wasm_data: Range<usize>,
address_map_data: Range<usize>,
trap_data: Range<usize>,
module: Arc<Module>,
funcs: PrimaryMap<DefinedFuncIndex, FunctionInfo>,
trampolines: Vec<Trampoline>,
funcs: PrimaryMap<DefinedFuncIndex, (WasmFunctionInfo, FunctionLoc)>,
trampolines: Vec<(SignatureIndex, FunctionLoc)>,
meta: Metadata,
code: Range<usize>,
code_memory: CodeMemory,
code_memory: Arc<CodeMemory>,
dbg_jit_registration: Option<GdbJitImageRegistration>,
/// A unique ID used to register this module with the engine.
unique_id: CompiledModuleId,
func_names: Vec<FunctionName>,
func_name_data: Range<usize>,
/// Map of dwarf sections indexed by `gimli::SectionId` which points to the
/// range within `code_memory`'s mmap as to the contents of the section.
dwarf_sections: Vec<Range<usize>>,
}
impl CompiledModule {
/// Creates `CompiledModule` directly from a precompiled artifact.
///
/// The `mmap` argument is expecte to be the result of a previous call to
/// `finish_compile` above. This is an ELF image, at this time, which
/// contains all necessary information to create a `CompiledModule` from a
/// compilation.
/// The `code_memory` argument is expected to be the result of a previous
/// call to `ObjectBuilder::finish` above. This is an ELF image, at this
/// time, which contains all necessary information to create a
/// `CompiledModule` from a compilation.
///
/// This method also takes `info`, an optionally-provided deserialization of
/// the artifacts' compilation metadata section. If this information is not
/// provided (e.g. it's set to `None`) then the information will be
/// This method also takes `info`, an optionally-provided deserialization
/// of the artifacts' compilation metadata section. If this information is
/// not provided then the information will be
/// deserialized from the image of the compilation artifacts. Otherwise it
/// will be assumed to be what would otherwise happen if the section were to
/// be deserialized.
/// will be assumed to be what would otherwise happen if the section were
/// to be deserialized.
///
/// The `profiler` argument here is used to inform JIT profiling runtimes
/// about new code that is loaded.
pub fn from_artifacts(
mmap: MmapVec,
mut info: Option<CompiledModuleInfo>,
code_memory: Arc<CodeMemory>,
info: CompiledModuleInfo,
profiler: &dyn ProfilingAgent,
id_allocator: &CompiledModuleIdAllocator,
) -> Result<Self> {
use gimli::SectionId::*;
// Parse the `code_memory` as an object file and extract information
// about where all of its sections are located, stored into the
// `CompiledModule` created here.
//
// Note that dwarf sections here specifically are those that are carried
// over directly from the original wasm module's dwarf sections, not the
// wasmtime-generated host DWARF sections.
let obj = File::parse(&mmap[..]).context("failed to parse internal elf file")?;
let mut wasm_data = None;
let mut address_map_data = None;
let mut func_name_data = None;
let mut trap_data = None;
let mut code = None;
let mut dwarf_sections = Vec::new();
for section in obj.sections() {
let name = section.name()?;
let data = section.data()?;
let range = subslice_range(data, &mmap);
let mut gimli = |id: gimli::SectionId| {
let idx = id as usize;
if dwarf_sections.len() <= idx {
dwarf_sections.resize(idx + 1, 0..0);
}
dwarf_sections[idx] = range.clone();
};
match name {
ELF_WASM_DATA => wasm_data = Some(range),
ELF_WASMTIME_ADDRMAP => address_map_data = Some(range),
ELF_WASMTIME_TRAPS => trap_data = Some(range),
ELF_NAME_DATA => func_name_data = Some(range),
".text" => code = Some(range),
// Parse the metadata if it's not already available
// in-memory.
ELF_WASMTIME_INFO => {
if info.is_none() {
info = Some(
bincode::deserialize(data)
.context("failed to deserialize wasmtime module info")?,
);
}
}
// Register dwarf sections into the `dwarf_sections`
// array which is indexed by `gimli::SectionId`
".debug_abbrev.wasm" => gimli(DebugAbbrev),
".debug_addr.wasm" => gimli(DebugAddr),
".debug_aranges.wasm" => gimli(DebugAranges),
".debug_frame.wasm" => gimli(DebugFrame),
".eh_frame.wasm" => gimli(EhFrame),
".eh_frame_hdr.wasm" => gimli(EhFrameHdr),
".debug_info.wasm" => gimli(DebugInfo),
".debug_line.wasm" => gimli(DebugLine),
".debug_line_str.wasm" => gimli(DebugLineStr),
".debug_loc.wasm" => gimli(DebugLoc),
".debug_loc_lists.wasm" => gimli(DebugLocLists),
".debug_macinfo.wasm" => gimli(DebugMacinfo),
".debug_macro.wasm" => gimli(DebugMacro),
".debug_pub_names.wasm" => gimli(DebugPubNames),
".debug_pub_types.wasm" => gimli(DebugPubTypes),
".debug_ranges.wasm" => gimli(DebugRanges),
".debug_rng_lists.wasm" => gimli(DebugRngLists),
".debug_str.wasm" => gimli(DebugStr),
".debug_str_offsets.wasm" => gimli(DebugStrOffsets),
".debug_types.wasm" => gimli(DebugTypes),
".debug_cu_index.wasm" => gimli(DebugCuIndex),
".debug_tu_index.wasm" => gimli(DebugTuIndex),
_ => log::debug!("ignoring section {name}"),
}
}
let info = info.ok_or_else(|| anyhow!("failed to find wasm info section"))?;
let mut ret = Self {
module: Arc::new(info.module),
funcs: info.funcs,
trampolines: info.trampolines,
wasm_data: wasm_data.ok_or_else(|| anyhow!("missing wasm data section"))?,
address_map_data: address_map_data.unwrap_or(0..0),
func_name_data: func_name_data.unwrap_or(0..0),
trap_data: trap_data.ok_or_else(|| anyhow!("missing trap data section"))?,
code: code.ok_or_else(|| anyhow!("missing code section"))?,
dbg_jit_registration: None,
code_memory: CodeMemory::new(mmap),
code_memory,
meta: info.meta,
unique_id: id_allocator.alloc(),
func_names: info.func_names,
dwarf_sections,
};
ret.code_memory
.publish()
.context("failed to publish code memory")?;
ret.register_debug_and_profiling(profiler)?;
Ok(ret)
@@ -505,8 +444,8 @@ impl CompiledModule {
fn register_debug_and_profiling(&mut self, profiler: &dyn ProfilingAgent) -> Result<()> {
// Register GDB JIT images; initialize profiler and load the wasm module.
if self.meta.native_debug_info_present {
let code = self.code();
let bytes = create_gdbjit_image(self.mmap().to_vec(), (code.as_ptr(), code.len()))
let text = self.text();
let bytes = create_gdbjit_image(self.mmap().to_vec(), (text.as_ptr(), text.len()))
.map_err(SetupError::DebugInfo)?;
profiler.module_load(self, Some(&bytes));
let reg = GdbJitImageRegistration::register(bytes);
@@ -529,33 +468,16 @@ impl CompiledModule {
self.code_memory.mmap()
}
/// Returns the concatenated list of all data associated with this wasm
/// module.
///
/// This is used for initialization of memories and all data ranges stored
/// in a `Module` are relative to the slice returned here.
pub fn wasm_data(&self) -> &[u8] {
&self.mmap()[self.wasm_data.clone()]
}
/// Returns the encoded address map section used to pass to
/// `wasmtime_environ::lookup_file_pos`.
pub fn address_map_data(&self) -> &[u8] {
&self.mmap()[self.address_map_data.clone()]
}
/// Returns the encoded trap information for this compiled image.
///
/// For more information see `wasmtime_environ::trap_encoding`.
pub fn trap_data(&self) -> &[u8] {
&self.mmap()[self.trap_data.clone()]
/// Returns the underlying owned mmap of this compiled image.
pub fn code_memory(&self) -> &Arc<CodeMemory> {
&self.code_memory
}
/// Returns the text section of the ELF image for this compiled module.
///
/// This memory should have the read/execute permissions.
pub fn code(&self) -> &[u8] {
&self.mmap()[self.code.clone()]
pub fn text(&self) -> &[u8] {
self.code_memory.text()
}
/// Return a reference-counting pointer to a module.
@@ -574,7 +496,7 @@ impl CompiledModule {
// `from_utf8_unchecked` if we really wanted since this section is
// guaranteed to only have valid utf-8 data. Until it's a problem it's
// probably best to double-check this though.
let data = &self.mmap()[self.func_name_data.clone()];
let data = self.code_memory().func_name_data();
Some(str::from_utf8(&data[name.offset as usize..][..name.len as usize]).unwrap())
}
@@ -588,9 +510,9 @@ impl CompiledModule {
pub fn finished_functions(
&self,
) -> impl ExactSizeIterator<Item = (DefinedFuncIndex, *const [VMFunctionBody])> + '_ {
let code = self.code();
self.funcs.iter().map(move |(i, info)| {
let func = &code[info.start as usize..][..info.length as usize];
let text = self.text();
self.funcs.iter().map(move |(i, (_, loc))| {
let func = &text[loc.start as usize..][..loc.length as usize];
(
i,
std::ptr::slice_from_raw_parts(func.as_ptr().cast::<VMFunctionBody>(), func.len()),
@@ -600,15 +522,15 @@ impl CompiledModule {
/// Returns the per-signature trampolines for this module.
pub fn trampolines(&self) -> impl Iterator<Item = (SignatureIndex, VMTrampoline, usize)> + '_ {
let code = self.code();
self.trampolines.iter().map(move |info| {
let text = self.text();
self.trampolines.iter().map(move |(signature, loc)| {
(
info.signature,
*signature,
unsafe {
let ptr = &code[info.start as usize];
let ptr = &text[loc.start as usize];
std::mem::transmute::<*const u8, VMTrampoline>(ptr)
},
info.length as usize,
loc.length as usize,
)
})
}
@@ -623,7 +545,7 @@ impl CompiledModule {
) -> impl Iterator<Item = (*const [VMFunctionBody], &[StackMapInformation])> {
self.finished_functions()
.map(|(_, f)| f)
.zip(self.funcs.values().map(|f| f.stack_maps.as_slice()))
.zip(self.funcs.values().map(|f| &f.0.stack_maps[..]))
}
/// Lookups a defined function by a program counter value.
@@ -631,14 +553,14 @@ impl CompiledModule {
/// Returns the defined function index and the relative address of
/// `text_offset` within the function itself.
pub fn func_by_text_offset(&self, text_offset: usize) -> Option<(DefinedFuncIndex, u32)> {
let text_offset = text_offset as u64;
let text_offset = u32::try_from(text_offset).unwrap();
let index = match self
.funcs
.binary_search_values_by_key(&text_offset, |info| {
debug_assert!(info.length > 0);
.binary_search_values_by_key(&text_offset, |(_, loc)| {
debug_assert!(loc.length > 0);
// Return the inclusive "end" of the function
info.start + u64::from(info.length) - 1
loc.start + loc.length - 1
}) {
Ok(k) => {
// Exact match, pc is at the end of this function
@@ -652,22 +574,33 @@ impl CompiledModule {
}
};
let body = self.funcs.get(index)?;
let start = body.start;
let end = body.start + u64::from(body.length);
let (_, loc) = self.funcs.get(index)?;
let start = loc.start;
let end = loc.start + loc.length;
if text_offset < start || end < text_offset {
return None;
}
Some((index, (text_offset - body.start) as u32))
Some((index, text_offset - loc.start))
}
/// Gets the function location information for a given function index.
pub fn func_loc(&self, index: DefinedFuncIndex) -> &FunctionLoc {
&self
.funcs
.get(index)
.expect("defined function should be present")
.1
}
/// Gets the function information for a given function index.
pub fn func_info(&self, index: DefinedFuncIndex) -> &FunctionInfo {
self.funcs
pub fn wasm_func_info(&self, index: DefinedFuncIndex) -> &WasmFunctionInfo {
&self
.funcs
.get(index)
.expect("defined function should be present")
.0
}
/// Creates a new symbolication context which can be used to further
@@ -681,12 +614,7 @@ impl CompiledModule {
return Ok(None);
}
let dwarf = gimli::Dwarf::load(|id| -> Result<_> {
let range = self
.dwarf_sections
.get(id as usize)
.cloned()
.unwrap_or(0..0);
let data = &self.mmap()[range];
let data = self.code_memory().dwarf_section(id);
Ok(EndianSlice::new(data, gimli::LittleEndian))
})?;
let cx = addr2line::Context::from_dwarf(dwarf)
@@ -709,7 +637,7 @@ impl CompiledModule {
/// If this function returns `false` then `lookup_file_pos` will always
/// return `None`.
pub fn has_address_map(&self) -> bool {
!self.address_map_data().is_empty()
!self.code_memory.address_map_data().is_empty()
}
/// Returns the bounds, in host memory, of where this module's compiled

5
crates/jit/src/lib.rs
View File

@@ -27,10 +27,9 @@ mod instantiate;
mod profiling;
mod unwind;
pub use crate::code_memory::{CodeMemory, ELF_WASM_BTI};
pub use crate::code_memory::CodeMemory;
pub use crate::instantiate::{
finish_compile, mmap_vec_from_obj, subslice_range, CompiledModule, CompiledModuleInfo,
SetupError, SymbolizeContext,
subslice_range, CompiledModule, CompiledModuleInfo, ObjectBuilder, SetupError, SymbolizeContext,
};
pub use demangling::*;
pub use profiling::*;

6
crates/jit/src/unwind/systemv.rs
View File

@@ -14,6 +14,8 @@ extern "C" {
}
impl UnwindRegistration {
pub const SECTION_NAME: &str = ".eh_frame";
/// Registers precompiled unwinding information with the system.
///
/// The `_base_address` field is ignored here (only used on other
@@ -67,10 +69,6 @@ impl UnwindRegistration {
Ok(UnwindRegistration { registrations })
}
pub fn section_name() -> &'static str {
".eh_frame"
}
}
impl Drop for UnwindRegistration {

6
crates/jit/src/unwind/winx64.rs
View File

@@ -10,6 +10,8 @@ pub struct UnwindRegistration {
}
impl UnwindRegistration {
pub const SECTION_NAME: &str = ".pdata";
pub unsafe fn new(
base_address: *const u8,
unwind_info: *const u8,
@@ -31,10 +33,6 @@ impl UnwindRegistration {
functions: unwind_info as usize,
})
}
pub fn section_name() -> &'static str {
".pdata"
}
}
impl Drop for UnwindRegistration {