This commit adds a `wasmtime-rust` crate to the `misc` folder next to
the previously added Python extension. The intention is that this
showcases loading a WebAssembly file natively in Rust and how with an
attribute macro it can feel lightweight in terms of boilerplate.
The macro itself is pretty non-featureful today beyond the bare bones to
get anything working, but there's all sorts of possibilities like
JIT-compiled entry stubs we could eventually do with all the type
information!
This commit adds initial support for [WebAssembly Interface
Types][proposal] to wasmtime. This is all intended to be quite
experimental, so experimental in fact that even the name of the
[proposal] is still in flux. (this has otherwise been known as "host
bindings" or "webidl bindings" or "wasm bindings").
The goal of this commit is to start adding support the wasmtime set of
crates for WebAssembly Interface Types. A new `wasmtime-interface-types`
crate has been added with very basic support for dynamically invoking
and inspecting the various bindings of a module. This is in turn powered
by the `wasm-webidl-bindings` crate which is shared with the
`wasm-bindgen` CLI tool as a producer of this section.
Currently the only integration in `wasmtime`-the-binary itself is that
when passed the `--invoke` argument the CLI will now attempt to invoke
the target function with arguments as parsed from the command line
itself. For example if you export a function like:
fn render(&str) -> String
Then passing `--invoke render` will require one argument on the command
line, which is the first argument as a string, and the return value is
printed to the console. This differs from today's interpretation of
`--invoke` where it is a failure if the invoked function takes more than
one argument and the return values are currently ignored.
This is intended to also be the basis of embedding wasmtime in other
contexts which also want to consume WebAssembly interface types. A
Python extension is also added to this repository which implements the
`wasmtime` package on PyPI. This Python extension is intended to make it
as easy as `pip3 install wasmtime` to load a WebAssembly file with
WebAssembly Interface Types into Python. Extensions for other languages
is of course possible as well!
One of the major missing pieces from this is handling imported functions
with interface bindings. Currently the embedding support doesn't have
much ability to support handling imports ergonomically, so it's intended
that this will be included in a follow-up patch.
[proposal]: https://github.com/webassembly/webidl-bindings
Co-authored-by: Yury Delendik <ydelendik@mozilla.com>
Move `src/*.rs` to `src/bin/*.rs` which are automatically inferred as
binaries and move `src/utils.rs` to `src/lib.rs` which is compiled as a
reusable library for each of the binaries we're building.
Currently our Linux binaries aren't quite as portable as they otherwise could
be. There's two primary reasons for this, and one of them is that the binary is
produced in a relatively recent Linux distribution (Ubuntu 16.04) which means
it has a relatively recent requirement in terms of glibc versions. On
OSX/Windows we can set some flags to rely on older libc implementations, but on
Linux we have to actually build against an older version.
This commit switches the container for the build to CentOS 6 instead of the
default Ubuntu 16.04. The main trick here is also finding a C++11-capable
compiler to compile wabt. Turns out though there's a helpful tutorial for this
at https://edwards.sdsu.edu/research/c11-on-centos-6/ and it was as easy as
installing a few packages.
The second portability concern of our Linux binaries is that they link
dynamically to `libstdc++.so` which isn't always installed on target systems,
or even if it is it may be too old or have a different ABI. This is solved by
statically linking to `libstdc++.a` in the build on Azure by doing a bit of
trickery with libraries and what's available.
After these results the glibc requirements drops from 2.18 (released in 2013)
to 2.6 (released in 2007) and avoids the need for users to have libstdc++.so
installed. We may eventually want to investigate fully-static musl binaries,
but finding a musl compiler for C++ is something I'm not that good at, so I
figure this is probably good enough for now.
* Remove cmake/bindgen/llvm from wasmtime-runtime
This commit removes the cmake/bindgen dependency (which removes the need
for `llvm-config`) from the `wasmtime-runtime` crate. The C++ code is
instead compiled with the `cc` crate (it's just one file anyway) and the
interface is handwritten since it's quite small anyway.
Some other changes are:
* The `TrapContext` type in C++ was removed since it was unused, and it
was moved to Rust with a `Cell` on each field.
* Functions between Rust/C++ now return `int` instead of `bool` to make
them a bit more FFI compatible portably.
* The `jmp_buf` type has a workaround that will be fixed in the next commit.
* Move setjmp/longjmp to C++
This commit moves the definition of setjmp and longjmp into C++. This is
primarily done because it's [debatable whether it's possible to call
`setjmp` from Rust][rfc]. The semantics of `setjmp` are that it returns
twice but LLVM doesn't actually know about this because rustc isn't
telling LLVM this information, so it's unclear whether it can ever be
safe.
Additionally this removes the need for Rust code to know the definition
of `jmp_buf` which is a pretty hairy type to define in Rust across
platforms.
The solution in this commit is to move all setjmp/longjmp code to C++,
and that way we should be able to guarantee that jumps over wasm JIT
code should always go from C++ to C++, removing Rust from the equation
for now from needing to get any fiddly bits working across platforms.
This should overall help it be a bit more portable and also means Rust
doesn't have to know about `jmp_buf` as a type.
The previous `Vec` of `jmp_buf` is now replaced with one thread-local
pointer where previous values are stored on the stack and restored when
the function returns. This is intended to be functionally the same as
the previous implementation.
[rfc]: https://github.com/rust-lang/rfcs/issues/2625
* rustfmt
* Use volatile loads/stores
* Remove mention of cmake from README
While the "__wasi_memory" name is something we considered, the name
currently being used for the memory exported to WASI is "memory", so
adjust the error message accordingly.
* Simple module compilation cache
* Fix base64 encoding bug
* Use warn! everywhere in cache system
* Remove unused import
* Temporary workaround for long path on Windows
* Remove unused import for non-windows builds
* Add command line argument to enable cache system + apply minor review feedback
* Initial implementation of partial module hashing
* Proper module hashing for the cache
* Use newer version of cranelift
This commit compiles all Rust code for the Windows release with
`-Ctarget-feature=+crt-static`. This is targeted at increasing the
binary compatibility of the binaries built to not rely on DLLs that
aren't always installed on Windows. Notably this statically links the C
runtime (notably used by the C++ code) and means that the final binary
relies on fewer dlls.
This in theory means that the binaries are only limited by the number of
APIs they use from Windows. Note that this also matches how we build
releases of Rust for MSVC.
* Set MACOSX_DEPLOYMENT_TARGET for macOS releases
This is an effort to ideally produce "more portable" binaries for the
releases we publish to GitHub. Currently the way macOS works is that
you're generally only guaranteed to work on the same platform you built
on and later (although it may sometimes work on older platforms). By
configuring this environment variable it should be possible to lower the
binary compatibility requirement, allowing running binaries on older OS
releases than the build machine is running.
I've chosen 10.9 here since it seems to be the lowest that "just works",
but there's no particular reason other than that for choosing this. Rust
itself chooses 10.8 (I think) for the compiler and 10.7 for the standard
library. This decision is largely driven by the C++ code from wabt-sys
which has more requirements about binary compatibility than Rust code
does.
Note that I don't actually have older macOS machines to test on as well,
but I can at least confirm that this does affect the build process!
* Comment the env var added
* Refactor Azure Pipelines config and tweak releases
* Extract out doc/rustfmt jobs into their own separate builders. Helps
avoiding having to skip tons of steps and can get failing results more
quickly.
* Extract out Rust installation logic to a dedicated template.
* Separate out the build/test job matrices, where one matrix runs tests
and another runs a full build
* Refactor release directory structure to follow a convention where
`foo.tar.gz` extracts to a folder called `foo` and follow unix-like
conventions and copy over the license/readme files into the release
tarballs.
* Swap order of build/test
This Azure Pipelines setup compiles and tests Wasmtime for Linux, macOS, and Windows.
If the CI run was triggered by a new tag being created, a new release for that tag is created with a changelog relative to the last tag release and archives of the builds for all platforms.
If the CI run was triggered by new commits landing on `master`, the release `latest-master` is updated with a new changelog relative to the last tag release and archives of the new builds for all platforms.
Note: This PR also contains changes to disable a bunch of tests on Windows, which are failing due to issues with signal handling.
This change adds an `--enable-simd` flag to the binaries in this project. This allows the ISA `enable_simd` flag to be set and to configure the validation configuration used by wasmparser to allow SIMD instructions.
* Simple module compilation cache
* Fix base64 encoding bug
* Use warn! everywhere in cache system
* Remove unused import
* Temporary workaround for long path on Windows
* Remove unused import for non-windows builds
* Cache directory hierarchy
* Fix conditional compilation for debug mode
* Minor enhancements
* Simple module compilation cache
* Fix base64 encoding bug
* Use warn! everywhere in cache system
* Remove unused import
* Temporary workaround for long path on Windows
* Remove unused import for non-windows builds
* Add command line argument to enable cache system + apply minor review feedback
