This patch adds initial support for ittapi which is an open
source profiling api for instrumentation and tracing and profiling
of jitted code. Result files can be read by VTune for analysis
Build:
cargo build --features=vtune
Profile: // Using amplxe-cl from VTune
amplxe-cl -v -collect hostpost target/debug/wasmtime --vtune test.wasm
* Remove the wasmtime Python extension from this repo
This commit removes the `crates/misc/py` folder and all associated
doo-dads like CI. This module has been rewritten to use the C API
natively and now lives at
https://github.com/bytecodealliance/wasmtime-py as discussed on #1390
Both cranelift-codegen and wasmtime-debug need to map Cranelift registers to Gimli registers. Previously both crates had an almost-identical `map_reg` implementation. This change:
- removes the wasmtime-debug implementation
- improves the cranelift-codegen implementation with custom errors
- exposes map_reg in `cranelift_codegen::isa::fde::map_reg` and subsequently `wasmtime_environ::isa::fde::map_reg`
This fills out a few functions related to frame information in the C
API. This additionally adds some extension functions to get the module
name and function name from the C API.
* Move back to only one WASI submodule
This commit fixes the issue where we have two WASI submodules for build
reasons in this repository. The fix was to place the submodule in the
`wasi-common` crate, and then anyone using the `wig` crate has to be
sure to define a `WASI_ROOT` env var in a build script to be able to
parse witx files.
With all that in place `wasi-common` becomes the source of truth for the
witx files we're parsing, and crates like `wasmtime-wasi` use
build-scripts shenanigans to read the same witx files. This should
hopefully get us so we're compatible with publishing and still only have
one submodule!
* rustfmt
This commit augments `wiggle` with trace log generation for the shims,
returned errno values, and returned values proper (if any, i.e.,
different than unit type `()`). What that means is that every syscall
will have auto-generated up to 3 traces, for instance,
```
TRACE wasi_common::wasi::wasi_snapshot_preview1 > fd_prestat_get(fd=Fd(3))
TRACE wasi_common::wasi::wasi_snapshot_preview1 > | result=(buf=Dir(PrestatDir { pr_name_len: 1 }))
TRACE wasi_common::wasi::wasi_snapshot_preview1 > | errno=No error occurred. System call completed successfully. (Errno::Success(0))
```
Putting logging behind a feature gate in this case means that the log calls
are generated by the `wiggle` crate regardless if the client requested
the feature or not, however, then their usage in the client lib is
dictated by the presence of the feature flag. So, for instance, `wasi-common`
has this feature enabled by default, while any other client lib
using `wiggle` if they don't want tracing enabled, they will just
leave the feature off. I'm not sure if this is what we wanted
but seemed easiest to implement quickly. Lemme y'all know your thoughts
about this!
Add some crate features to compile out support for these features of the
C API. Avoiding these two features if they're not necessary shaves about
2MB off the final shared object in some local tests!
* Refactor and improve safety of C API
This commit is intended to be a relatively large refactoring of the C
API which is targeted at improving the safety of our C API definitions.
Not all of the APIs have been updated yet but this is intended to be the
start.
The goal here is to make as many functions safe as we can, expressing
inputs/outputs as native Rust types rather than raw pointers wherever
possible. For example instead of `*const wasm_foo_t` we'd take
`&wasm_foo_t`. Instead of returning `*mut wasm_foo_t` we'd return
`Box<wasm_foo_t>`. No ABI/API changes are intended from this commit,
it's supposed to only change how we define all these functions
internally.
This commit also additionally implements a few more API bindings for
exposed vector types by unifying everything into one macro.
Finally, this commit moves many internal caches in the C API to the
`OnceCell` type which provides a safe interface for one-time
initialization.
* Split apart monolithic C API `lib.rs`
This commit splits the monolithic `src/lib.rs` in the C API crate into
lots of smaller files. The goal here is to make this a bit more readable
and digestable. Each module now contains only API bindings for a
particular type, roughly organized around the grouping in the wasm.h
header file already.
A few more extensions were added, such as filling out `*_as_*`
conversions with both const and non-const versions. Additionally many
APIs were made safer in the same style as the previous commit, generally
preferring Rust types rather than raw pointer types.
Overall no functional change is intended here, it should be mostly just
code movement and minor refactorings!
* Make a few wasi C bindings safer
Use safe Rust types where we can and touch up a few APIs here and there.
* Implement `wasm_*type_as_externtype*` APIs
This commit restructures `wasm_externtype_t` to be similar to
`wasm_extern_t` so type conversion between the `*_extern_*` variants to
the concrete variants are all simple casts. (checked in the case of
general to concrete, of course).
* Consistently imlpement host info functions in the API
This commit adds a small macro crate which is then used to consistently
define the various host-info-related functions in the C API. The goal
here is to try to mirror what the `wasm.h` header provides to provide a
full implementation of the header.
* Refactor use of Refs and RefMuts in wasi-common
This commit refactors the use of `Ref`s and `RefMut`s in `wasi-common`.
Now, `Entry` is stored behind an `Rc` inside the `EntryTable`. The `Entry`
itself on the other hand now stores rights behind a `RefCell` and the
descriptor as `Rc<RefCell<..>>` combo to enable easy reference tracking
and interior mutability which is required down the line in a couple of
syscalls. In essence, this implies that we no longer have need for
mutable accessor to `Entry` from `WasiCtx`, and so all related methods
go away (`get_entry_mut`, etc.).
While here, I've also simplified handling and aggregating of rights on
the `Entry` object. Instead of storing base and inheriting rights as
separate fields, they are now aggregated into one struct `EntryRights`
which features convenient constructors for each possible combination; i.e.,
when only base rights are set, or both base and inheriting are set, or
both are left as empty. Since we do need to be able to mutate those
rights down the line in `fdstat_set_rights` syscall, this object
is kept behind a `RefCell` (note no `Rc` since we don't need to pass it
around anywhere).
The descriptor field in `Entry` is now kept behind `Rc<RefCell<..>>` combo
since we not only need to mutate it down the line, but we also need to
be able to pass it around (as part of the machinery making `poll_oneoff`
work).
I've also removed `as_file` and `try_clone` methods on `Descriptor` struct
since they were adding more noise than necessary, and making them work
with `Rc` was unnecessarily complicated.
Finally, I've converted the `get_dir_from_os_handle` function into a
method attached to the `OsHandle` itself, called `dir_stream`. IMHO,
it makes more sense to have it there directly as a method than as a separate
function.
* Use Cell for types that are Copy
* Bump Wasmtime to 0.14.0.
* Update the publish script for the wiggle crate wiggle.
* More fixes.
* Fix lightbeam depenency version.
* cargo update
* Cargo update wasi-tests too.
And add cargo update to the version-bump scripts.
* Publishing fixes.
* Make WASI a symlink.
* More fixes.
* Cargo doesn't allow dev-dependencies to have optional features.
* Remove the symlink.
* Add WASI as another git submodule.
* Shuffle around the wiggle crates
This commit reorganizes the wiggle crates slightly by performing the
following transforms:
* The `crates/wiggle` crate, previously named `wiggle`, was moved to
`crates/wiggle/crates/macro` and is renamed to `wiggle-macro`.
* The `crates/wiggle/crates/runtime` crate, previously named
`wiggle-runtime`, was moved to `crates/wiggle` and is renamed to
`wiggle`.
* The new `wiggle` crate depends on `wiggle-macro` and reexports the macro.
The goal here is that consumers only deal with the `wiggle` crate
itself. No more crates depend on `wiggle-runtime` and all dependencies
are entirely on just the `wiggle` crate.
* Remove the `crates/wiggle/crates` directory
Move everything into `crates/wiggle` directly, like `wasi-common`
* Add wiggle-macro to test-all script
* Fixup a test
This commit adds the `wasmtime_func_new_with_env` C API function and refactors
the implementation to share the implementation between the C API and Wasmtime
extension variants.
This commit reimplements the C# API in terms of a Wasmtime linker.
It removes the custom binding implementation that was based on reflection in
favor of the linker's implementation.
This should make the C# API a little closer to the Rust API.
The `Engine` and `Store` types have been hidden behind the `Host` type which is
responsible for hosting WebAssembly module instances.
Documentation and tests have been updated.
This commit adds support for snapshot0 in the WASI C API.
A name parameter was added to `wasi_instance_new` to accept which WASI module
is being instantiated.
Additionally, the C# API now supports constructing a WASI instance based on the
WASI module name.
Fixes#1221.
* Expand the C API slightly
Fill out missing `wasm_{memory,table}_type` APIs and don't panic on the
`wasm_externtype_as_*type_const` APIs and opt for returning a null
pointer instead.
* Update crates/c-api/src/lib.rs
Co-Authored-By: Yury Delendik <ydelendik@mozilla.com>
Co-authored-by: Yury Delendik <ydelendik@mozilla.com>
* wiggle: emit a metadata module containing witx document
* wiggle: put metadata module behind a wiggle_metadata feature
* wasi-common: add wiggle_metadata feature and optional witx dep
* refactor according to alex's advice
* wasi-common: make snapshots pub
* wasi-common: i do need a wiggle_metadata feature to be available
* Tweak features and such
* wiggle: fix tests by passing metadata flag to wiggle-runtime
* wiggle: need to move wiggle-runtime to a non-dev dependency
so that the feature resolves for external users of the crates
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Clean up the use of mutable Entry
Until now, several syscalls including `fd_pwrite` etc. were relying on
mutating `&mut Entry` by mutating its inner file handle. This is
unnecessary in almost all cases since all methods mutating `std::fs::File`
in Rust's libstd are also implemented for `&std::fs::File`.
While here, I've also modified `OsHandle` in BSD to include `RefCell<Option<Dir>>`
rather than `Option<Mutex<Dir>>` as was until now. While `RefCell`
could easily be replaced with `RefCell`, since going multithreading
will require a lot of (probably even) conceptual changes to `wasi-common`,
I thought it'd be best not to mix single- with multithreading contexts
and swap all places at once when it comes to it.
I've also had to make some modifications to virtual FS which mainly
swapped mutability for interior mutability in places.
* Use one-liners wherever convenient
* Use `Linker` in `*.wast` testing
By default `Linker` disallows shadowing previously defined items, but it
looks like the `*.wast` test suites rely on this so this commit adds a
boolean flag to `Linker` as well indicating whether duplicates are
allowed.
* Review comments
* Add a test with a number of recursive instances
* Deny warnings in doctests
* No tabs
* Fix instructions for building rust modules in python examples
When I ran `rustc +nightly ...` the compiler just looked for a source file
called `+nightly`. I changed these instructions to use rustup + rustc instead.
* Initial documentation for python users
Added documentation for using the Wasmtime loader in python, and explained the
first two examples in the repo. Changed the import example to demonstrate
working with module linear memory.
* Fix include in python guide
* Wording
* Clarify memory usage
* Flow through the example better
* More word choice
* Make rustup a prereq
* Fix source code paths in python guide
* Fix rustup example in python guide
Co-Authored-By: Samrat Man Singh <samratmansingh@gmail.com>
* Replace command examples with preformat blocks
* Revert "Fix instructions for building rust modules in python examples"
This reverts commit 1738888a2df4e15aba1e26c8ef42058e7a2053bb.
* Left a block quote in a preformat example
Co-authored-by: Samrat Man Singh <samratmansingh@gmail.com>
* Add a `wasmtime::Linker` type
This commit adds a new type to the `wasmtime` crate, a `Linker`. This
linker is intended to vastly simplify calling `Instance::new` by easily
performing name resolution and incrementally defining state over time.
The goal here is to start down a path of making linking wasm modules in
`wasmtime` a first-class and ergonomic operation. This is highly likely
to evolve over time and get tweaked through releases as we iterate
towards a design well-suited for `wasmtime`, but this is intended to at
least be the initial foundation for such functionality.
This commit additionally also adds a C API for the linker and switches
the existing linking examples to using this linker in both Rust and C.
One piece of future work I'd like to tackle next is to integrate WASI
into the `wasmtime` crate in a more first-class manner. This [`Linker`]
type provides a great location to hook into the instantiation process to
easily instantiate modules with WASI imports. That's a relatively large
refactoring for now though and I figured it'd be best left for a
different time.
Closes#727
The crates/debug/src/transform/address_transform.rs is unoptimized in terms of data structures. This PR refactors this file to remove creation of intermediate in-heap structures, thus improves overall performance of the DWARF transformation.
* Reduce amount of memory allocated in translate_ranges_raw
* refactor translate_ranges
* Don't transform non-unit .debug_line
* type annotation for TransformRangeXXXIter's
* Fix empty generated wasm positions
Some structs and unions are large enough that making them `Copy` isn't
ideal. wasi-common only needed `Copy` in a few places that were easy to
fix. `SubscriptionClock` is 32 bytes, so it's not a bad a idea to pass
it by reference anyway.
* Add a .gitattributes file specifying LF-style line endings.
This is similar to [Rust's .gitattributes file] though simplified.
Most of our source and documentation files already used LF-style line
endings, including *.cs files, so this makes things more consistent.
[Rust's .gitattributes file]: https://github.com/rust-lang/rust/blob/master/.gitattributes
* Remove UTF-8 BOMs in *.cs files.
Most of our *.cs files don't have UTF-8 BOMs, so this makes things more
consistent.
* Add examples of linking and WASI
This commit adds two example programs, one for linking two modules
together and one for instantiating WASI. The linkage example
additionally uses WASI to get some meaningful output at this time.
cc #1272
* Add examples to the book as well
* More links!
* Ignore examples from rustdoc testsing
* More example updates
* More ignored
with the prev approach, it would be passed by reference sometimes
(e.g. when used as an Array argument) but by value most of the time.
this was inconsistient.
theres no need to pass the owned version, all operations are &self.
