* Leverage Cargo's workspace inheritance feature
This commit is an attempt to reduce the complexity of the Cargo
manifests in this repository with Cargo's workspace-inheritance feature
becoming stable in Rust 1.64.0. This feature allows specifying fields in
the root workspace `Cargo.toml` which are then reused throughout the
workspace. For example this PR shares definitions such as:
* All of the Wasmtime-family of crates now use `version.workspace =
true` to have a single location which defines the version number.
* All crates use `edition.workspace = true` to have one default edition
for the entire workspace.
* Common dependencies are listed in `[workspace.dependencies]` to avoid
typing the same version number in a lot of different places (e.g. the
`wasmparser = "0.89.0"` is now in just one spot.
Currently the workspace-inheritance feature doesn't allow having two
different versions to inherit, so all of the Cranelift-family of crates
still manually specify their version. The inter-crate dependencies,
however, are shared amongst the root workspace.
This feature can be seen as a method of "preprocessing" of sorts for
Cargo manifests. This will help us develop Wasmtime but shouldn't have
any actual impact on the published artifacts -- everything's dependency
lists are still the same.
* Fix wasi-crypto tests
* Update to cap-std 0.26.
This is primarily to pull in bytecodealliance/cap-std#271, the fix for #4936,
compilation on Rust nightly on Windows.
It also updates to rustix 0.35.10, to pull in bytecodealliance/rustix#403,
the fix for bytecodealliance/rustix#402, compilation on newer versions of
the libc crate, which changed a public function from `unsafe` to safe.
Fixes#4936.
* Update the system-interface audit for 0.23.
* Update the libc supply-chain config version.
* Optimize the WASI `random_get` implementation.
Use `StdRng` instead of the `OsRng` in the default implementation of
`random_get`. This uses a userspace CSPRNG, making `random_get` 3x faster
in simple benchmarks.
* Update cargo-vet audits for cap-std 0.25.3.
* Update all cap-std packages to 0.25.3.
This commit replaces #4869 and represents the actual version bump that
should have happened had I remembered to bump the in-tree version of
Wasmtime to 1.0.0 prior to the branch-cut date. Alas!
* Implement the remaining socket-related WASI functions.
The original WASI specification included `sock_read`, `sock_write`, and
`shutdown`. Now that we have some sockets support, implement these
additional functions, to make it easier for people porting existing code
to WASI.
It's expected that this will all be subsumed by the wasi-sockets
proposal, but for now, this is a relatively small change which should
hopefully unblock people trying to use the current `accept` support.
* Update to system-interface 0.22, which has fixes for Windows.
* Make wasi-common-std-sync's dependency on system-interface private.
Change some `pub` functions which exposed system-interface types to be
non-`pub`.
And, change `from_sysif_fdflags` functions to `get_fd_flags` functions
that take `impl AsFilelike` arguments instead of system-interface types.
With these changes, system-interface is no longer exposed in the
public API.
* Add a public API for `is_read_write` too.
Implementors using types implementing `AsFilelike` may want to use the
same `is_read_write` logic, without explicitly depending on
system-interface, so provide a function that provides that.
POSIX specifies that functions like `nanosleep` use the REALTIME clock,
so allow WASI `poll_oneoff` calls to use the REALTIME clock, at least
for non-absolute intervals. POSIX specifies that the timeouts should not
be affected by subsequent `clock_settime` calls, so they behave the same
way as MONOTONIC clock requests, so we can implement them as monotonic
requests.
* Stop returning `NOTCAPABLE` errors from WASI calls.
`ENOTCAPABLE` was an error code that is used as part of the rights
system, from CloudABI. There is a set of flags associated with each file
descriptor listing which operations can be performed with the file
descriptor, and if an attempt is made to perform an operation with a
file descriptor that isn't permitted by its rights flags, it fails with
`ENOTCAPABLE`.
WASI is removing the rights system. For example, WebAssembly/wasi-libc#294
removed support for translating `ENOTCAPABLE` into POSIX error codes, on
the assumption that engines should stop using it.
So as another step to migrating away from the rights system, remove uses
of the `ENOTCAPABLE` error.
* Update crates/wasi-common/src/file.rs
Co-authored-by: Jamey Sharp <jamey@minilop.net>
* Update crates/wasi-common/src/dir.rs
Co-authored-by: Jamey Sharp <jamey@minilop.net>
Co-authored-by: Jamey Sharp <jamey@minilop.net>
* Don't try to report file size or timestamps for stdio streams.
Calling `File::metadata()` on a stdio stream handle fails on Windows, where
the stdio streams are not files.
This `File::metadata()` call was effectively only being used to add file size
and timestamps to the result of `filestat_get`. It's common for users to
redirect stdio streams to interesting places, and applications
generally shouldn't change their behavior depending on the size or
timestamps of the file, if the streams are redirected to a file, so just
leave these fields to 0, which is commonly understood to represent
"unknown".
Fixes#4497.
* Update tracing-core to a version which doesn't depend on lazy-static.
* Update crossbeam-utils to a version that doesn't depend on lazy-static.
* Update crossbeam-epoch to a version that doesn't depend on lazy-static.
* Update clap to a version that doesn't depend on lazy-static.
* Convert Wasmtime's own use of lazy_static to once_cell.
* Make `GDB_REGISTRATION`'s comment a doc comment.
* Fix compilation on Windows.
This updates to rustix 0.35.6, and updates wasi-common to use cap-std 0.25 and
windows-sys (instead of winapi).
Changes include:
- Better error code mappings on Windows.
- Fixes undefined references to `utimensat` on Darwin.
- Fixes undefined references to `preadv64` and `pwritev64` on Android.
- Updates to io-lifetimes 0.7, which matches the io_safety API in Rust.
- y2038 bug fixes for 32-bit platforms
Relevant to Wasmtime, this fixes undefined references to `utimensat` and
`futimens` on macOS 10.12 and earlier. See bytecodealliance/rustix#157
for details.
It also contains a fix for s390x which isn't currently needed by Wasmtime
itself, but which is needed to make rustix's own testsuite pass on s390x,
which helps people packaging rustix for use in Wasmtime. See
bytecodealliance/rustix#277 for details.
* Upgrade all crates to the Rust 2021 edition
I've personally started using the new format strings for things like
`panic!("some message {foo}")` or similar and have been upgrading crates
on a case-by-case basis, but I think it probably makes more sense to go
ahead and blanket upgrade everything so 2021 features are always
available.
* Fix compile of the C API
* Fix a warning
* Fix another warning
* Bump to 0.36.0
* Add a two-week delay to Wasmtime's release process
This commit is a proposal to update Wasmtime's release process with a
two-week delay from branching a release until it's actually officially
released. We've had two issues lately that came up which led to this proposal:
* In #3915 it was realized that changes just before the 0.35.0 release
weren't enough for an embedding use case, but the PR didn't meet the
expectations for a full patch release.
* At Fastly we were about to start rolling out a new version of Wasmtime
when over the weekend the fuzz bug #3951 was found. This led to the
desire internally to have a "must have been fuzzed for this long"
period of time for Wasmtime changes which we felt were better
reflected in the release process itself rather than something about
Fastly's own integration with Wasmtime.
This commit updates the automation for releases to unconditionally
create a `release-X.Y.Z` branch on the 5th of every month. The actual
release from this branch is then performed on the 20th of every month,
roughly two weeks later. This should provide a period of time to ensure
that all changes in a release are fuzzed for at least two weeks and
avoid any further surprises. This should also help with any last-minute
changes made just before a release if they need tweaking since
backporting to a not-yet-released branch is much easier.
Overall there are some new properties about Wasmtime with this proposal
as well:
* The `main` branch will always have a section in `RELEASES.md` which is
listed as "Unreleased" for us to fill out.
* The `main` branch will always be a version ahead of the latest
release. For example it will be bump pre-emptively as part of the
release process on the 5th where if `release-2.0.0` was created then
the `main` branch will have 3.0.0 Wasmtime.
* Dates for major versions are automatically updated in the
`RELEASES.md` notes.
The associated documentation for our release process is updated and the
various scripts should all be updated now as well with this commit.
* Add notes on a security patch
* Clarify security fixes shouldn't be previewed early on CI
* Update to rustix 0.33.5, to fix a link error on Android
This updates to rustix 0.33.5, which includes bytecodealliance/rustix#258,
which fixes bytecodealliance/rustix#256, a link error on Android.
Fixes#3965.
* Bump the rustix versions in the Cargo.toml files too.
Currently, the use of the downcast method means that you have to use one
of the hard-coded types. But Enarx needs to define its own `WasiFile`
implementations. This works fine, except the resulting files cannot be
used in poll because they aren't part of the hard-coded list.
Replace this with an accessor method for the pollable type in
`WasiFile`. Because we provide a default implementation of the method
and manually implement it on all the hard-coded types, this is backwards
compatible.
Signed-off-by: Nathaniel McCallum <nathaniel@profian.com>
1. This makes it easier for implementors to deal with internal APIs.
2. This matches the signatures of the WASI Snapshot traits.
Although it is likely true that these methods would have to become
immutable in order to implement threading efficiently, threading will
impact a large number of existing traits. So this change is practical
for now with an already-unavoidable change required for threading.
Signed-off-by: Nathaniel McCallum <nathaniel@profian.com>
Other than doc updates, this just contains bytecodealliance/cap-std#235,
a fix for compilation errors on Rust nightly that look like this:
```
error[E0308]: mismatched types
--> cap-primitives/src/fs/via_parent/rename.rs:22:58
|
22 | let (old_dir, old_basename) = open_parent(old_start, &old_path)?;
| ^^^^^^^^^ expected struct `Path`, found opaque type
|
::: cap-primitives/src/rustix/fs/dir_utils.rs:67:48
|
67 | pub(crate) fn strip_dir_suffix(path: &Path) -> impl Deref<Target = Path> + '_ {
| ------------------------------ the found opaque type
|
= note: expected struct `Path`
found opaque type `impl Deref<Target = Path>`
```
Following up on #3696, use the new is-terminal crate to test for a tty
rather than having platform-specific logic in Wasmtime. The is-terminal
crate has a platform-independent API which takes a handle.
This also updates the tree to cap-std 0.24 etc., to avoid depending on
multiple versions of io-lifetimes at once, as enforced by the cargo deny
check.
With the addition of `sock_accept()` in `wasi-0.11.0`, wasmtime can now
implement basic networking for pre-opened sockets.
For Windows `AsHandle` was replaced with `AsRawHandleOrSocket` to cope
with the duality of Handles and Sockets.
For Unix a `wasi_cap_std_sync::net::Socket` enum was created to handle
the {Tcp,Unix}{Listener,Stream} more efficiently in
`WasiCtxBuilder::preopened_socket()`.
The addition of that many `WasiFile` implementors was mainly necessary,
because of the difference in the `num_ready_bytes()` function.
A known issue is Windows now busy polling on sockets, because except
for `stdin`, nothing is querying the status of windows handles/sockets.
Another know issue on Windows, is that there is no crate providing
support for `fcntl(fd, F_GETFL, 0)` on a socket.
Signed-off-by: Harald Hoyer <harald@profian.com>
WASI doesn't have an `isatty` ioctl or syscall, so wasi-libc's `isatty`
implementation uses the file descriptor type and rights to determine if
the file descriptor is likely to be a tty. The real fix here will be to
add an `isatty` call to WASI. But for now, have Wasmtime set the
filetype and rights for file descriptors so that wasi-libc's `isatty`
works as expected.
`ptr::cast` has the advantage of being unable to silently cast
`*const T` to `*mut T`. This turned up several places that were
performing such casts, which this PR also fixes.
* Update to cap-std 0.22.0.
The main change relevant to Wasmtime here is that this includes the
rustix fix for compilation errors on Rust nightly with the `asm!` macro.
* Add itoa to deny.toml.
* Update the doc and fuzz builds to the latest Rust nightly.
* Update to libc 0.2.112 to pick up the `POLLRDHUP` fix.
* Update to cargo-fuzz 0.11, for compatibility with Rust nightly.
This appears to be the fix for rust-fuzz/cargo-fuzz#277.
This pulls in a fix for Android, where Android's seccomp policy on older
versions is to make `openat2` irrecoverably crash the process, so we have
to do a version check up front rather than relying on `ENOSYS` to
determine if `openat2` is supported.
And it pulls in the fix for the link errors when multiple versions of
rsix/rustix are linked in.
And it has updates for two crate renamings: rsix has been renamed to
rustix, and unsafe-io has been renamed to io-extras.
* Adjust dependency directives between crates
This commit is a preparation for the release process for Wasmtime. The
specific changes here are to delineate which crates are "public", and
all version requirements on non-public crates will now be done with
`=A.B.C` version requirements instead of today's `A.B.C` version
requirements.
The purpose for doing this is to assist with patch releases that might
happen in the future. Patch releases of wasmtime are already required to
not break the APIs of "public" crates, but no such guarantee is given
about "internal" crates. This means that a patch release runs the risk,
for example, of breaking an internal API. In doing so though we would
also need to release a new major version of the internal crate, but we
wouldn't have a great hole in the number scheme of major versions to do
so. By using `=A.B.C` requirements for internal crates it means we can
safely ignore strict semver-compatibility between releases of internal
crates for patch releases, since the only consumers of the crate will be
the corresponding patch release of the `wasmtime` crate itself (or other
public crates).
The `publish.rs` script has been updated with a check to verify that
dependencies on internal crates are all specified with an `=`
dependency, and dependnecies on all public crates are without a `=`
dependency. This will hopefully make it so we don't have to worry about
what to use where, we just let CI tell us what to do. Using this
modification all version dependency declarations have been updated.
Note that some crates were adjusted to simply remove their `version`
requirement in cases such as the crate wasn't published anyway (`publish
= false` was specified) or it's in the `dev-dependencies` section which
doesn't need version specifiers for path dependencies.
* Switch to normal sever deps for cranelift dependencies
These crates will now all be considered "public" where in patch releases
they will be guaranteed to not have breaking changes.
* Use rsix to make system calls in Wasmtime.
`rsix` is a system call wrapper crate that we use in `wasi-common`,
which can provide the following advantages in the rest of Wasmtime:
- It eliminates some `unsafe` blocks in Wasmtime's code. There's
still an `unsafe` block in the library, but this way, the `unsafe`
is factored out and clearly scoped.
- And, it makes error handling more consistent, factoring out code for
checking return values and `io::Error::last_os_error()`, and code that
does `errno::set_errno(0)`.
This doesn't cover *all* system calls; `rsix` doesn't implement
signal-handling APIs, and this doesn't cover calls made through `std` or
crates like `userfaultfd`, `rand`, and `region`.
- Fixes for compiling on OpenBSD
- io-lifetimes 0.3.0 has an option (io_lifetimes_use_std, which is off
by default) for testing the `io_safety` feature in Rust nightly.
* Port wasi-common to io-lifetimes.
This ports wasi-common from unsafe-io to io-lifetimes.
Ambient authority is now indicated via calls to `ambient_authority()`
from the ambient-authority crate, rather than using `unsafe` blocks.
The `GetSetFdFlags::set_fd_flags` function is now split into two phases,
to simplify lifetimes in implementations which need to close and re-open
the underlying file.
* Use posish for errno values instead of libc.
This eliminates one of the few remaining direct libc dependencies.
* Port to posish::io::poll.
Use posish::io::poll instead of calling libc directly. This factors out
more code from Wasmtime, and eliminates the need to manipulate raw file
descriptors directly.
And, this eliminates the last remaining direct dependency on libc in
wasi-common.
* Port wasi-c-api to io-lifetimes.
* Update to posish 0.16.0.
* Embeded NULs in filenames now get `EINVAL` instead of `EILSEQ`.
* Accept either `EILSEQ` or `EINVAL` for embedded NULs.
* Bump the nightly toolchain to 2021-07-12.
This fixes build errors on the semver crate, which as of this writing
builds with latest nightly and stable but not 2021-04-11, the old pinned
version.
* Have cap-std-sync re-export ambient_authority so that users get the same version.
