# Overview
This commit makes changes to the `wiggle::from_witx` procedural in order
to allow for escaping strict and reserved Rust keywords.
Additionally, this commit introduces the ability to use a `witx_literal`
field in the `{..}` object provided as an argument to
`wiggle::from_witx`. This field allows for witx documents to be provided
as inline string literals.
Documentation comments are added to the methods of
`wiggle_generate::names::Names` struct responsible for generating
`proc_macro2::Ident` words.
## Keyword Escaping
Today, an interface that includes witx identifiers that conflict with
with Rust syntax will cause the `from_witx` macro to panic at
compilation time.
Here is a small example (adapted from
`/crates/wiggle/tests/keywords.rs`) that demonstrates this issue:
```
;; Attempts to define a module `self`, containing a trait `Self`. Both
;; of these are reserved keywords, and will thus cause a compilation
;; error.
(module $self
(@interface func (export "betchya_cant_implement_this")
)
)
```
Building off of code that (as of `master` today)
[demonstrates a strategy][esc] for escaping keywords, we introduce an
internal `escaping` module to `generate/src/config.rs` that contains
code responsible for escaping Rust keywords in a generalized manner.
[esc]: 0dd77d36f8/crates/wiggle/generate/src/names.rs (L106)
Some code related to special cases, such as accounting for
[`errno::2big`][err] while generating names for enum variants, is moved
into this module as well.
[err]: https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md#-errno-enumu16
As mentioned in the document comments of this diff, we do not include
weak keywords like `'static` or `union`. Their semantics do not impact
us in the same way from a code generation perspective.
## witx_literal
First, some background. Trait names, type names, and so on use a
camel-cased naming convention. As such, `Self` is the only keyword that
can potentially conflict with these identifiers. (See the [Rust
Reference][key] for a complete list of strict, reserved, and weak
keywords.)
When writing tests, this meant that many tests had to be outlined into
separate files, as items with the name `$self` could not be defined in
the same namespace. As such, it seemed like a worthwhile feature to
implement while the above work was being developed.
The most important function to note is the `load_document` inherent
method added to `WitxConf`, and that `WitxConf` is now an enum
containing either (a) a collection of paths, identical to its current
functionality, or (b) a single string literal.
Note that a witx document given to `from_witx` using a string literal
provided to `from_witx` cannot include `use (..)` directives, per
the `witx::parse` documentation.
(See: https://docs.rs/witx/0.8.5/witx/fn.parse.html)
Two newtypes, `Paths` and `Literal`, are introduced to facilitate the
parsing of `WitxConf` values. Their public API and trait implementations
has been kept to the minimum required to satisfy compilation in order to
limit the scope of this diff. Additional surface for external consumers
can be added in follow-up commits if deemed necessary in review.
* Add AArch64 tests to CI
This commit enhances our CI with an AArch64 builder. Currently we have
no physical hardware to run on so for now we run all tests in an
emulator. The AArch64 build is cross-compiled from x86_64 from Linux.
Tests all happen in release mode with a recent version of QEMU (recent
version because it's so much faster, and in release mode because debug
mode tests take quite a long time in an emulator).
The goal here was not to get all tests passing on CI, but rather to get
AArch64 running on CI and get it green at the same time. To achieve that
goal many tests are now ignored on aarch64 platforms. Many tests fail
due to unimplemented functionality in the aarch64 backend (#1521), and
all wasmtime tests involving compilation are also disabled due to
panicking attempting to generate generate instruction offset information
for trap symbolication (#1523).
Despite this, though, all Cranelift tests and other wasmtime tests
should be runnin on AArch64 through QEMU with this PR. Additionally
we'll have an AArch64 binary release of Wasmtime for Linux, although it
won't be too useful just yet since it will panic on almost all wasm
modules.
* Review comments
* wiggle-generate: paramaterize library on module path to runtime
This change makes no functional difference to users who only use the
wiggle crate.
Add a parameter to the `Names` constructor that determines the module
that runtime components (e.g. GuestPtr, GuestError etc) of wiggle come
from. For `wiggle` users this is just `quote!(wiggle)`, but other
libraries which consume wiggle-generate may wrap and re-export wiggle
under some other path, and not want their consumers to have to know
about the wiggle dependency, e.g. `quote!(my_crate::some_path::wiggle)`.
* wiggle-generate,macro: move more logic into macro
better for code reuse elsewhere
* Implement interrupting wasm code, reimplement stack overflow
This commit is a relatively large change for wasmtime with two main
goals:
* Primarily this enables interrupting executing wasm code with a trap,
preventing infinite loops in wasm code. Note that resumption of the
wasm code is not a goal of this commit.
* Additionally this commit reimplements how we handle stack overflow to
ensure that host functions always have a reasonable amount of stack to
run on. This fixes an issue where we might longjmp out of a host
function, skipping destructors.
Lots of various odds and ends end up falling out in this commit once the
two goals above were implemented. The strategy for implementing this was
also lifted from Spidermonkey and existing functionality inside of
Cranelift. I've tried to write up thorough documentation of how this all
works in `crates/environ/src/cranelift.rs` where gnarly-ish bits are.
A brief summary of how this works is that each function and each loop
header now checks to see if they're interrupted. Interrupts and the
stack overflow check are actually folded into one now, where function
headers check to see if they've run out of stack and the sentinel value
used to indicate an interrupt, checked in loop headers, tricks functions
into thinking they're out of stack. An interrupt is basically just
writing a value to a location which is read by JIT code.
When interrupts are delivered and what triggers them has been left up to
embedders of the `wasmtime` crate. The `wasmtime::Store` type has a
method to acquire an `InterruptHandle`, where `InterruptHandle` is a
`Send` and `Sync` type which can travel to other threads (or perhaps
even a signal handler) to get notified from. It's intended that this
provides a good degree of flexibility when interrupting wasm code. Note
though that this does have a large caveat where interrupts don't work
when you're interrupting host code, so if you've got a host import
blocking for a long time an interrupt won't actually be received until
the wasm starts running again.
Some fallout included from this change is:
* Unix signal handlers are no longer registered with `SA_ONSTACK`.
Instead they run on the native stack the thread was already using.
This is possible since stack overflow isn't handled by hitting the
guard page, but rather it's explicitly checked for in wasm now. Native
stack overflow will continue to abort the process as usual.
* Unix sigaltstack management is now no longer necessary since we don't
use it any more.
* Windows no longer has any need to reset guard pages since we no longer
try to recover from faults on guard pages.
* On all targets probestack intrinsics are disabled since we use a
different mechanism for catching stack overflow.
* The C API has been updated with interrupts handles. An example has
also been added which shows off how to interrupt a module.
Closes#139Closes#860Closes#900
* Update comment about magical interrupt value
* Store stack limit as a global value, not a closure
* Run rustfmt
* Handle review comments
* Add a comment about SA_ONSTACK
* Use `usize` for type of `INTERRUPTED`
* Parse human-readable durations
* Bring back sigaltstack handling
Allows libstd to print out stack overflow on failure still.
* Add parsing and emission of stack limit-via-preamble
* Fix new example for new apis
* Fix host segfault test in release mode
* Fix new doc example
* [wiggle]: wiggle-generate internal interface for Names simplified
im using it in other libraries where I don't want to construct a
wiggle_generate::Config just to use the ctx_type out of it.
* wiggle: define_func can get trait name as argument
this flexibility needed for some customization over in lucet
* Wasmtime 0.15.0 and Cranelift 0.62.0. (#1398)
* Bump more ad-hoc versions.
* Add build.rs to wasi-common's Cargo.toml.
* Update the env var name in more places.
* Remove a redundant echo.
* Move back to only one WASI submodule
This commit fixes the issue where we have two WASI submodules for build
reasons in this repository. The fix was to place the submodule in the
`wasi-common` crate, and then anyone using the `wig` crate has to be
sure to define a `WASI_ROOT` env var in a build script to be able to
parse witx files.
With all that in place `wasi-common` becomes the source of truth for the
witx files we're parsing, and crates like `wasmtime-wasi` use
build-scripts shenanigans to read the same witx files. This should
hopefully get us so we're compatible with publishing and still only have
one submodule!
* rustfmt
This commit augments `wiggle` with trace log generation for the shims,
returned errno values, and returned values proper (if any, i.e.,
different than unit type `()`). What that means is that every syscall
will have auto-generated up to 3 traces, for instance,
```
TRACE wasi_common::wasi::wasi_snapshot_preview1 > fd_prestat_get(fd=Fd(3))
TRACE wasi_common::wasi::wasi_snapshot_preview1 > | result=(buf=Dir(PrestatDir { pr_name_len: 1 }))
TRACE wasi_common::wasi::wasi_snapshot_preview1 > | errno=No error occurred. System call completed successfully. (Errno::Success(0))
```
Putting logging behind a feature gate in this case means that the log calls
are generated by the `wiggle` crate regardless if the client requested
the feature or not, however, then their usage in the client lib is
dictated by the presence of the feature flag. So, for instance, `wasi-common`
has this feature enabled by default, while any other client lib
using `wiggle` if they don't want tracing enabled, they will just
leave the feature off. I'm not sure if this is what we wanted
but seemed easiest to implement quickly. Lemme y'all know your thoughts
about this!
* Bump Wasmtime to 0.14.0.
* Update the publish script for the wiggle crate wiggle.
* More fixes.
* Fix lightbeam depenency version.
* cargo update
* Cargo update wasi-tests too.
And add cargo update to the version-bump scripts.
* Shuffle around the wiggle crates
This commit reorganizes the wiggle crates slightly by performing the
following transforms:
* The `crates/wiggle` crate, previously named `wiggle`, was moved to
`crates/wiggle/crates/macro` and is renamed to `wiggle-macro`.
* The `crates/wiggle/crates/runtime` crate, previously named
`wiggle-runtime`, was moved to `crates/wiggle` and is renamed to
`wiggle`.
* The new `wiggle` crate depends on `wiggle-macro` and reexports the macro.
The goal here is that consumers only deal with the `wiggle` crate
itself. No more crates depend on `wiggle-runtime` and all dependencies
are entirely on just the `wiggle` crate.
* Remove the `crates/wiggle/crates` directory
Move everything into `crates/wiggle` directly, like `wasi-common`
* Add wiggle-macro to test-all script
* Fixup a test
* wiggle: emit a metadata module containing witx document
* wiggle: put metadata module behind a wiggle_metadata feature
* wasi-common: add wiggle_metadata feature and optional witx dep
* refactor according to alex's advice
* wasi-common: make snapshots pub
* wasi-common: i do need a wiggle_metadata feature to be available
* Tweak features and such
* wiggle: fix tests by passing metadata flag to wiggle-runtime
* wiggle: need to move wiggle-runtime to a non-dev dependency
so that the feature resolves for external users of the crates
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Some structs and unions are large enough that making them `Copy` isn't
ideal. wasi-common only needed `Copy` in a few places that were easy to
fix. `SubscriptionClock` is 32 bytes, so it's not a bad a idea to pass
it by reference anyway.
with the prev approach, it would be passed by reference sometimes
(e.g. when used as an Array argument) but by value most of the time.
this was inconsistient.
theres no need to pass the owned version, all operations are &self.
