T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Use an mmap-friendly serialization format (#3257)

Browse Source 
* Use an mmap-friendly serialization format

This commit reimplements the main serialization format for Wasmtime's
precompiled artifacts. Previously they were generally a binary blob of
`bincode`-encoded metadata prefixed with some versioning information.
The downside of this format, though, is that loading a precompiled
artifact required pushing all information through `bincode`. This is
inefficient when some data, such as trap/address tables, are rarely
accessed.

The new format added in this commit is one which is designed to be
`mmap`-friendly. This means that the relevant parts of the precompiled
artifact are already page-aligned for updating permissions of pieces
here and there. Additionally the artifact is optimized so that if data
is rarely read then we can delay reading it until necessary.

The new artifact format for serialized modules is an ELF file. This is
not a public API guarantee, so it cannot be relied upon. In the meantime
though this is quite useful for exploring precompiled modules with
standard tooling like `objdump`. The ELF file is already constructed as
part of module compilation, and this is the main contents of the
serialized artifact.

THere is some extra information, though, not encoded in each module's
individual ELF file such as type information. This information continues
to be `bincode`-encoded, but it's intended to be much smaller and much
faster to deserialize. This extra information is appended to the end of
the ELF file. This means that the original ELF file is still a valid ELF
file, we just get to have extra bits at the end. More information on the
new format can be found in the module docs of the serialization module
of Wasmtime.

Another refatoring implemented as part of this commit is to deserialize
and store object files directly in `mmap`-backed storage. This avoids
the need to copy bytes after the artifact is loaded into memory for each
compiled module, and in a future commit it opens up the door to avoiding
copying the text section into a `CodeMemory`. For now, though, the main
change is that copies are not necessary when loading from a precompiled
compilation artifact once the artifact is itself in mmap-based memory.

To assist with managing `mmap`-based memory a new `MmapVec` type was
added to `wasmtime_jit` which acts as a form of `Vec<T>` backed by a
`wasmtime_runtime::Mmap`. This type notably supports `drain(..N)` to
slice the buffer into disjoint regions that are all separately owned,
such as having a separately owned window into one artifact for all
object files contained within.

Finally this commit implements a small refactoring in `wasmtime-cache`
to use the standard artifact format for cache entries rather than a
bincode-encoded version. This required some more hooks for
serializing/deserializing but otherwise the crate still performs as
before.

* Review comments
This commit is contained in:
Alex Crichton
2021-08-30 09:19:20 -05:00
committed by GitHub
parent 16854e73c5
commit c73be1f13a
10 changed files with 684 additions and 263 deletions
Download Patch File Download Diff File Expand all files Collapse all files

278
crates/jit/src/instantiate.rs
View File

@@ -6,7 +6,7 @@
use crate::code_memory::CodeMemory;
use crate::debug::create_gdbjit_image;
use crate::link::link_module;
use crate::ProfilingAgent;
use crate::{MmapVec, ProfilingAgent};
use anyhow::{anyhow, Context, Result};
use object::read::File;
use object::write::{Object, StandardSegment};
@@ -68,15 +68,6 @@ pub enum SetupError {
DebugInfo(#[from] anyhow::Error),
}
/// Final result of compilation which supports serialization to disk.
#[derive(Serialize, Deserialize)]
pub struct CompilationArtifacts {
// NB: this structure is in a transitionary phase and will soon go away. At
// this time it only contains the ELF image created by compilation, and in
// the near future even this will be removed.
obj: Box<[u8]>,
}
/// Secondary in-memory results of compilation.
///
/// This opaque structure can be optionally passed back to
@@ -113,125 +104,120 @@ struct Metadata {
has_wasm_debuginfo: bool,
}
impl CompilationArtifacts {
/// Finishes compilation of the `translation` specified, producing the final
/// compilation artifacts and auxiliary information.
///
/// This function will consume the final results of compiling a wasm module
/// and finish the ELF image in-progress as part of `obj` by appending any
/// compiler-agnostic sections.
///
/// The auxiliary `CompiledModuleInfo` structure returned here has also been
/// serialized into `CompilationArtifacts`, but if the caller will quickly
/// turn-around and invoke `CompiledModule::from_artifacts` after this then
/// the information can be passed to that method to avoid extra
/// deserialization. This is done to avoid a serialize-then-deserialize for
/// API calls like `Module::new` where the compiled module is immediately
/// going to be used.
pub fn new(
translation: ModuleTranslation<'_>,
mut obj: Object,
funcs: PrimaryMap<DefinedFuncIndex, FunctionInfo>,
tunables: &Tunables,
) -> Result<(CompilationArtifacts, CompiledModuleInfo)> {
let ModuleTranslation {
mut module,
debuginfo,
/// Finishes compilation of the `translation` specified, producing the final
/// compilation artifact and auxiliary information.
///
/// This function will consume the final results of compiling a wasm module
/// and finish the ELF image in-progress as part of `obj` by appending any
/// compiler-agnostic sections.
///
/// The auxiliary `CompiledModuleInfo` structure returned here has also been
/// serialized into the object returned, but if the caller will quickly
/// turn-around and invoke `CompiledModule::from_artifacts` after this then the
/// information can be passed to that method to avoid extra deserialization.
/// This is done to avoid a serialize-then-deserialize for API calls like
/// `Module::new` where the compiled module is immediately going to be used.
///
/// The `MmapVec` returned here contains the compiled image and resides in
/// mmap'd memory for easily switching permissions to executable afterwards.
pub fn finish_compile(
translation: ModuleTranslation<'_>,
mut obj: Object,
funcs: PrimaryMap<DefinedFuncIndex, FunctionInfo>,
tunables: &Tunables,
) -> Result<(MmapVec, CompiledModuleInfo)> {
let ModuleTranslation {
mut module,
debuginfo,
has_unparsed_debuginfo,
data,
passive_data,
..
} = translation;
// Place all data from the wasm module into a section which will the
// source of the data later at runtime.
let data_id = obj.add_section(
obj.segment_name(StandardSegment::Data).to_vec(),
ELF_WASM_DATA.as_bytes().to_vec(),
SectionKind::ReadOnlyData,
);
let mut total_data_len = 0;
for data in data.iter() {
obj.append_section_data(data_id, data, 1);
total_data_len += data.len();
}
for data in passive_data.iter() {
obj.append_section_data(data_id, data, 1);
}
// Update passive data offsets since they're all located after the other
// data in the module.
for (_, range) in module.passive_data_map.iter_mut() {
range.start = range.start.checked_add(total_data_len as u32).unwrap();
range.end = range.end.checked_add(total_data_len as u32).unwrap();
}
// Insert the wasm raw wasm-based debuginfo into the output, if
// requested. Note that this is distinct from the native debuginfo
// possibly generated by the native compiler, hence these sections
// getting wasm-specific names.
if tunables.parse_wasm_debuginfo {
push_debug(&mut obj, &debuginfo.dwarf.debug_abbrev);
push_debug(&mut obj, &debuginfo.dwarf.debug_addr);
push_debug(&mut obj, &debuginfo.dwarf.debug_aranges);
push_debug(&mut obj, &debuginfo.dwarf.debug_info);
push_debug(&mut obj, &debuginfo.dwarf.debug_line);
push_debug(&mut obj, &debuginfo.dwarf.debug_line_str);
push_debug(&mut obj, &debuginfo.dwarf.debug_str);
push_debug(&mut obj, &debuginfo.dwarf.debug_str_offsets);
push_debug(&mut obj, &debuginfo.debug_ranges);
push_debug(&mut obj, &debuginfo.debug_rnglists);
}
// Encode a `CompiledModuleInfo` structure into the `ELF_WASMTIME_INFO`
// section of this image. This is not necessary when the returned module
// is never serialized to disk, which is also why we return a copy of
// the `CompiledModuleInfo` structure to the caller in case they don't
// want to deserialize this value immediately afterwards from the
// section. Otherwise, though, this is necessary to reify a `Module` on
// the other side from disk-serialized artifacts in
// `Module::deserialize` (a Wasmtime API).
let info_id = obj.add_section(
obj.segment_name(StandardSegment::Data).to_vec(),
ELF_WASMTIME_INFO.as_bytes().to_vec(),
SectionKind::ReadOnlyData,
);
let mut bytes = Vec::new();
let info = CompiledModuleInfo {
module,
funcs,
meta: Metadata {
native_debug_info_present: tunables.generate_native_debuginfo,
has_unparsed_debuginfo,
data,
passive_data,
..
} = translation;
code_section_offset: debuginfo.wasm_file.code_section_offset,
has_wasm_debuginfo: tunables.parse_wasm_debuginfo,
},
};
bincode::serialize_into(&mut bytes, &info)?;
obj.append_section_data(info_id, &bytes, 1);
// Place all data from the wasm module into a section which will the
// source of the data later at runtime.
let data_id = obj.add_section(
obj.segment_name(StandardSegment::Data).to_vec(),
ELF_WASM_DATA.as_bytes().to_vec(),
SectionKind::ReadOnlyData,
return Ok((MmapVec::from_obj(obj)?, info));
fn push_debug<'a, T>(obj: &mut Object, section: &T)
where
T: gimli::Section<gimli::EndianSlice<'a, gimli::LittleEndian>>,
{
let data = section.reader().slice();
if data.is_empty() {
return;
}
let section_id = obj.add_section(
obj.segment_name(StandardSegment::Debug).to_vec(),
wasm_section_name(T::id()).as_bytes().to_vec(),
SectionKind::Debug,
);
let mut total_data_len = 0;
for data in data.iter() {
obj.append_section_data(data_id, data, 1);
total_data_len += data.len();
}
for data in passive_data.iter() {
obj.append_section_data(data_id, data, 1);
}
// Update passive data offsets since they're all located after the other
// data in the module.
for (_, range) in module.passive_data_map.iter_mut() {
range.start = range.start.checked_add(total_data_len as u32).unwrap();
range.end = range.end.checked_add(total_data_len as u32).unwrap();
}
// Insert the wasm raw wasm-based debuginfo into the output, if
// requested. Note that this is distinct from the native debuginfo
// possibly generated by the native compiler, hence these sections
// getting wasm-specific names.
if tunables.parse_wasm_debuginfo {
push_debug(&mut obj, &debuginfo.dwarf.debug_abbrev);
push_debug(&mut obj, &debuginfo.dwarf.debug_addr);
push_debug(&mut obj, &debuginfo.dwarf.debug_aranges);
push_debug(&mut obj, &debuginfo.dwarf.debug_info);
push_debug(&mut obj, &debuginfo.dwarf.debug_line);
push_debug(&mut obj, &debuginfo.dwarf.debug_line_str);
push_debug(&mut obj, &debuginfo.dwarf.debug_str);
push_debug(&mut obj, &debuginfo.dwarf.debug_str_offsets);
push_debug(&mut obj, &debuginfo.debug_ranges);
push_debug(&mut obj, &debuginfo.debug_rnglists);
}
// Encode a `CompiledModuleInfo` structure into the `ELF_WASMTIME_INFO`
// section of this image. This is not necessary when the returned module
// is never serialized to disk, which is also why we return a copy of
// the `CompiledModuleInfo` structure to the caller in case they don't
// want to deserialize this value immediately afterwards from the
// section. Otherwise, though, this is necessary to reify a `Module` on
// the other side from disk-serialized artifacts in
// `Module::deserialize` (a Wasmtime API).
let info_id = obj.add_section(
obj.segment_name(StandardSegment::Data).to_vec(),
ELF_WASMTIME_INFO.as_bytes().to_vec(),
SectionKind::ReadOnlyData,
);
let mut bytes = Vec::new();
let info = CompiledModuleInfo {
module,
funcs,
meta: Metadata {
native_debug_info_present: tunables.generate_native_debuginfo,
has_unparsed_debuginfo,
code_section_offset: debuginfo.wasm_file.code_section_offset,
has_wasm_debuginfo: tunables.parse_wasm_debuginfo,
},
};
bincode::serialize_into(&mut bytes, &info)?;
obj.append_section_data(info_id, &bytes, 1);
return Ok((
CompilationArtifacts {
obj: obj.write()?.into(),
},
info,
));
fn push_debug<'a, T>(obj: &mut Object, section: &T)
where
T: gimli::Section<gimli::EndianSlice<'a, gimli::LittleEndian>>,
{
let data = section.reader().slice();
if data.is_empty() {
return;
}
let section_id = obj.add_section(
obj.segment_name(StandardSegment::Debug).to_vec(),
wasm_section_name(T::id()).as_bytes().to_vec(),
SectionKind::Debug,
);
obj.append_section_data(section_id, data, 1);
}
obj.append_section_data(section_id, data, 1);
}
}
@@ -270,7 +256,7 @@ pub struct CompiledModule {
wasm_data: Range<usize>,
address_map_data: Range<usize>,
trap_data: Range<usize>,
artifacts: CompilationArtifacts,
mmap: MmapVec,
module: Arc<Module>,
funcs: PrimaryMap<DefinedFuncIndex, FunctionInfo>,
meta: Metadata,
@@ -280,7 +266,12 @@ pub struct CompiledModule {
}
impl CompiledModule {
/// Creates `CompiledModule` directly from `CompilationArtifacts`.
/// Creates `CompiledModule` directly from a precompiled artifact.
///
/// The `mmap` argument is expecte to be the result of a previous call to
/// `finish_compile` above. This is an ELF image, at this time, which
/// contains all necessary information to create a `CompiledModule` from a
/// compilation.
///
/// This method also takes `info`, an optionally-provided deserialization of
/// the artifacts' compilation metadata section. If this information is not
@@ -292,11 +283,11 @@ impl CompiledModule {
/// The `profiler` argument here is used to inform JIT profiling runtimes
/// about new code that is loaded.
pub fn from_artifacts(
artifacts: CompilationArtifacts,
mmap: MmapVec,
info: Option<CompiledModuleInfo>,
profiler: &dyn ProfilingAgent,
) -> Result<Arc<Self>> {
let obj = File::parse(&artifacts.obj[..])
let obj = File::parse(&mmap[..])
.with_context(|| "failed to parse internal ELF compilation artifact")?;
let section = |name: &str| {
@@ -314,9 +305,9 @@ impl CompiledModule {
};
let module = Arc::new(info.module);
let funcs = info.funcs;
let wasm_data = subslice_range(section(ELF_WASM_DATA)?, &artifacts.obj);
let address_map_data = subslice_range(section(ELF_WASMTIME_ADDRMAP)?, &artifacts.obj);
let trap_data = subslice_range(section(ELF_WASMTIME_TRAPS)?, &artifacts.obj);
let wasm_data = subslice_range(section(ELF_WASM_DATA)?, &mmap);
let address_map_data = subslice_range(section(ELF_WASMTIME_ADDRMAP)?, &mmap);
let trap_data = subslice_range(section(ELF_WASMTIME_TRAPS)?, &mmap);
// Allocate all of the compiled functions into executable memory,
// copying over their contents.
@@ -336,7 +327,7 @@ impl CompiledModule {
meta: info.meta,
funcs,
module,
artifacts,
mmap,
wasm_data,
address_map_data,
trap_data,
@@ -357,7 +348,7 @@ impl CompiledModule {
// Register GDB JIT images; initialize profiler and load the wasm module.
let dbg_jit_registration = if self.meta.native_debug_info_present {
let bytes = create_gdbjit_image(
self.artifacts.obj.to_vec(),
self.mmap.to_vec(),
(
self.code.range.0 as *const u8,
self.code.range.1 - self.code.range.0,
@@ -376,9 +367,10 @@ impl CompiledModule {
Ok(())
}
/// Extracts `CompilationArtifacts` from the compiled module.
pub fn compilation_artifacts(&self) -> &CompilationArtifacts {
&self.artifacts
/// Returns the underlying memory which contains the compiled module's
/// image.
pub fn mmap(&self) -> &MmapVec {
&self.mmap
}
/// Returns the concatenated list of all data associated with this wasm
@@ -387,20 +379,20 @@ impl CompiledModule {
/// This is used for initialization of memories and all data ranges stored
/// in a `Module` are relative to the slice returned here.
pub fn wasm_data(&self) -> &[u8] {
&self.artifacts.obj[self.wasm_data.clone()]
&self.mmap[self.wasm_data.clone()]
}
/// Returns the encoded address map section used to pass to
/// `wasmtime_environ::lookup_file_pos`.
pub fn address_map_data(&self) -> &[u8] {
&self.artifacts.obj[self.address_map_data.clone()]
&self.mmap[self.address_map_data.clone()]
}
/// Returns the encoded trap information for this compiled image.
///
/// For more information see `wasmtime_environ::trap_encoding`.
pub fn trap_data(&self) -> &[u8] {
&self.artifacts.obj[self.trap_data.clone()]
&self.mmap[self.trap_data.clone()]
}
/// Return a reference-counting pointer to a module.
@@ -500,7 +492,7 @@ impl CompiledModule {
if !self.meta.has_wasm_debuginfo {
return Ok(None);
}
let obj = File::parse(&self.artifacts.obj[..])
let obj = File::parse(&self.mmap[..])
.context("failed to parse internal ELF file representation")?;
let dwarf = gimli::Dwarf::load(|id| -> Result<_> {
let data = obj
@@ -603,7 +595,7 @@ fn build_code_memory(
///
/// This method requires that `inner` is a sub-slice of `outer`, and if that
/// isn't true then this method will panic.
fn subslice_range(inner: &[u8], outer: &[u8]) -> Range<usize> {
pub fn subslice_range(inner: &[u8], outer: &[u8]) -> Range<usize> {
if inner.len() == 0 {
return 0..0;
}