T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Validate modules while translating (#2059)

Browse Source 
* Validate modules while translating

This commit is a change to cranelift-wasm to validate each function body
as it is translated. Additionally top-level module translation functions
will perform module validation. This commit builds on changes in
wasmparser to perform module validation interwtwined with parsing and
translation. This will be necessary for future wasm features such as
module linking where the type behind a function index, for example, can
be far away in another module. Additionally this also brings a nice
benefit where parsing the binary only happens once (instead of having an
up-front serial validation step) and validation can happen in parallel
for each function.

Most of the changes in this commit are plumbing to make sure everything
lines up right. The major functional change here is that module
compilation should be faster by validating in parallel (or skipping
function validation entirely in the case of a cache hit). Otherwise from
a user-facing perspective nothing should be that different.

This commit does mean that cranelift's translation now inherently
validates the input wasm module. This means that the Spidermonkey
integration of cranelift-wasm will also be validating the function as
it's being translated with cranelift. The associated PR for wasmparser
(bytecodealliance/wasmparser#62) provides the necessary tools to create
a `FuncValidator` for Gecko, but this is something I'll want careful
review for before landing!

* Read function operators until EOF

This way we can let the validator take care of any issues with
mismatched `end` instructions and/or trailing operators/bytes.
This commit is contained in:
Alex Crichton
2020-10-05 11:02:01 -05:00
committed by GitHub
parent df8f85f4bc
commit 2c6841041d
34 changed files with 709 additions and 790 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
crates/wasmtime/Cargo.toml
View File

@@ -15,8 +15,8 @@ wasmtime-environ = { path = "../environ", version = "0.20.0" }
wasmtime-jit = { path = "../jit", version = "0.20.0" }
wasmtime-cache = { path = "../cache", version = "0.20.0", optional = true }
wasmtime-profiling = { path = "../profiling", version = "0.20.0" }
wasmparser = "0.59.0"
target-lexicon = { version = "0.11.0", default-features = false }
wasmparser = "0.62.0"
anyhow = "1.0.19"
region = "2.2.0"
libc = "0.2"

2
crates/wasmtime/src/instance.rs
View File

@@ -38,7 +38,7 @@ fn instantiate(
let instance = store.add_instance(instance);
instance
.initialize(
config.wasm_bulk_memory,
config.features.bulk_memory,
&compiled_module.data_initializers(),
)
.map_err(|e| -> Error {

91
crates/wasmtime/src/module.rs
View File

@@ -4,6 +4,7 @@ use crate::types::{EntityType, ExportType, ExternType, ImportType};
use anyhow::{bail, Context, Result};
use std::path::Path;
use std::sync::{Arc, Mutex};
use wasmparser::Validator;
#[cfg(feature = "cache")]
use wasmtime_cache::ModuleCacheEntry;
use wasmtime_jit::{CompilationArtifacts, CompiledModule};
@@ -238,70 +239,6 @@ impl Module {
/// # }
/// ```
pub fn from_binary(engine: &Engine, binary: &[u8]) -> Result<Module> {
Module::validate(engine, binary)?;
// Note that the call to `from_binary_unchecked` here should be ok
// because we previously validated the binary, meaning we're guaranteed
// to pass a valid binary for `engine`.
unsafe { Module::from_binary_unchecked(engine, binary) }
}
/// Creates a new WebAssembly `Module` from the given in-memory `binary`
/// data, skipping validation and asserting that `binary` is a valid
/// WebAssembly module.
///
/// This function is the same as [`Module::new`] except that it skips the
/// call to [`Module::validate`] and it does not support the text format of
/// WebAssembly. The WebAssembly binary is not validated for
/// correctness and it is simply assumed as valid.
///
/// For more information about creation of a module and the `engine` argument
/// see the documentation of [`Module::new`].
///
/// # Unsafety
///
/// This function is `unsafe` due to the unchecked assumption that the input
/// `binary` is valid. If the `binary` is not actually a valid wasm binary it
/// may cause invalid machine code to get generated, cause panics, etc.
///
/// It is only safe to call this method if [`Module::validate`] succeeds on
/// the same arguments passed to this function.
///
/// # Errors
///
/// This function may fail for many of the same reasons as [`Module::new`].
/// While this assumes that the binary is valid it still needs to actually
/// be somewhat valid for decoding purposes, and the basics of decoding can
/// still fail.
pub unsafe fn from_binary_unchecked(engine: &Engine, binary: &[u8]) -> Result<Module> {
Module::compile(engine, binary)
}
/// Validates `binary` input data as a WebAssembly binary given the
/// configuration in `engine`.
///
/// This function will perform a speedy validation of the `binary` input
/// WebAssembly module (which is in [binary form][binary], the text format
/// is not accepted by this function) and return either `Ok` or `Err`
/// depending on the results of validation. The `engine` argument indicates
/// configuration for WebAssembly features, for example, which are used to
/// indicate what should be valid and what shouldn't be.
///
/// Validation automatically happens as part of [`Module::new`], but is a
/// requirement for [`Module::from_binary_unchecked`] to be safe.
///
/// # Errors
///
/// If validation fails for any reason (type check error, usage of a feature
/// that wasn't enabled, etc) then an error with a description of the
/// validation issue will be returned.
///
/// [binary]: https://webassembly.github.io/spec/core/binary/index.html
pub fn validate(engine: &Engine, binary: &[u8]) -> Result<()> {
engine.config().validator().validate_all(binary)?;
Ok(())
}
unsafe fn compile(engine: &Engine, binary: &[u8]) -> Result<Self> {
#[cfg(feature = "cache")]
let artifacts = ModuleCacheEntry::new("wasmtime", engine.cache_config())
.get_data((engine.compiler(), binary), |(compiler, binary)| {
@@ -323,6 +260,32 @@ impl Module {
})
}
/// Validates `binary` input data as a WebAssembly binary given the
/// configuration in `engine`.
///
/// This function will perform a speedy validation of the `binary` input
/// WebAssembly module (which is in [binary form][binary], the text format
/// is not accepted by this function) and return either `Ok` or `Err`
/// depending on the results of validation. The `engine` argument indicates
/// configuration for WebAssembly features, for example, which are used to
/// indicate what should be valid and what shouldn't be.
///
/// Validation automatically happens as part of [`Module::new`].
///
/// # Errors
///
/// If validation fails for any reason (type check error, usage of a feature
/// that wasn't enabled, etc) then an error with a description of the
/// validation issue will be returned.
///
/// [binary]: https://webassembly.github.io/spec/core/binary/index.html
pub fn validate(engine: &Engine, binary: &[u8]) -> Result<()> {
let mut validator = Validator::new();
validator.wasm_features(engine.config().features);
validator.validate_all(binary)?;
Ok(())
}
/// Serialize compilation artifacts to the buffer. See also `deseriaize`.
pub fn serialize(&self) -> Result<Vec<u8>> {
let artifacts = (

46
crates/wasmtime/src/runtime.rs
View File

@@ -12,7 +12,7 @@ use std::path::Path;
use std::rc::{Rc, Weak};
use std::sync::Arc;
use target_lexicon::Triple;
use wasmparser::Validator;
use wasmparser::WasmFeatures;
#[cfg(feature = "cache")]
use wasmtime_cache::CacheConfig;
use wasmtime_environ::settings::{self, Configurable, SetError};
@@ -45,11 +45,7 @@ pub struct Config {
pub(crate) profiler: Arc<dyn ProfilingAgent>,
pub(crate) memory_creator: Option<MemoryCreatorProxy>,
pub(crate) max_wasm_stack: usize,
wasm_threads: bool,
wasm_reference_types: bool,
pub(crate) wasm_bulk_memory: bool,
wasm_simd: bool,
wasm_multi_value: bool,
pub(crate) features: WasmFeatures,
}
impl Config {
@@ -98,11 +94,7 @@ impl Config {
profiler: Arc::new(NullProfilerAgent),
memory_creator: None,
max_wasm_stack: 1 << 20,
wasm_threads: false,
wasm_reference_types: cfg!(target_arch = "x86_64"),
wasm_bulk_memory: true,
wasm_simd: false,
wasm_multi_value: true,
features: WasmFeatures::default(),
}
}
@@ -167,7 +159,7 @@ impl Config {
///
/// [threads]: https://github.com/webassembly/threads
pub fn wasm_threads(&mut self, enable: bool) -> &mut Self {
self.wasm_threads = enable;
self.features.threads = enable;
// The threads proposal depends on the bulk memory proposal
if enable {
self.wasm_bulk_memory(true);
@@ -189,7 +181,7 @@ impl Config {
///
/// [proposal]: https://github.com/webassembly/reference-types
pub fn wasm_reference_types(&mut self, enable: bool) -> &mut Self {
self.wasm_reference_types = enable;
self.features.reference_types = enable;
self.flags
.set("enable_safepoints", if enable { "true" } else { "false" })
@@ -224,7 +216,7 @@ impl Config {
///
/// [proposal]: https://github.com/webassembly/simd
pub fn wasm_simd(&mut self, enable: bool) -> &mut Self {
self.wasm_simd = enable;
self.features.simd = enable;
let val = if enable { "true" } else { "false" };
self.flags
.set("enable_simd", val)
@@ -242,7 +234,7 @@ impl Config {
///
/// [proposal]: https://github.com/webassembly/bulk-memory-operations
pub fn wasm_bulk_memory(&mut self, enable: bool) -> &mut Self {
self.wasm_bulk_memory = enable;
self.features.bulk_memory = enable;
self
}
@@ -256,7 +248,7 @@ impl Config {
///
/// [proposal]: https://github.com/webassembly/multi-value
pub fn wasm_multi_value(&mut self, enable: bool) -> &mut Self {
self.wasm_multi_value = enable;
self.features.multi_value = enable;
self
}
@@ -615,19 +607,9 @@ impl Config {
self.isa_flags.clone().finish(settings::Flags::new(flags))
}
pub(crate) fn validator(&self) -> Validator {
let mut ret = Validator::new();
ret.wasm_threads(self.wasm_threads)
.wasm_bulk_memory(self.wasm_bulk_memory)
.wasm_multi_value(self.wasm_multi_value)
.wasm_reference_types(self.wasm_reference_types)
.wasm_simd(self.wasm_simd);
return ret;
}
fn build_compiler(&self) -> Compiler {
let isa = self.target_isa();
Compiler::new(isa, self.strategy, self.tunables.clone())
Compiler::new(isa, self.strategy, self.tunables.clone(), self.features)
}
/// Hashes/fingerprints compiler setting to ensure that compatible
@@ -666,11 +648,11 @@ impl fmt::Debug for Config {
f.debug_struct("Config")
.field("debug_info", &self.tunables.debug_info)
.field("strategy", &self.strategy)
.field("wasm_threads", &self.wasm_threads)
.field("wasm_reference_types", &self.wasm_reference_types)
.field("wasm_bulk_memory", &self.wasm_bulk_memory)
.field("wasm_simd", &self.wasm_simd)
.field("wasm_multi_value", &self.wasm_multi_value)
.field("wasm_threads", &self.features.threads)
.field("wasm_reference_types", &self.features.reference_types)
.field("wasm_bulk_memory", &self.features.bulk_memory)
.field("wasm_simd", &self.features.simd)
.field("wasm_multi_value", &self.features.multi_value)
.field(
"flags",
&settings::Flags::new(self.flags.clone()).to_string(),