* winch(fuzz): Initial support for differential fuzzing
This commit introduces initial support for differential fuzzing for Winch. In
order to fuzz winch, this change introduces the `winch` cargo feature. When the
`winch` cargo feature is enabled the differential fuzz target uses `wasmi` as
the differential engine and `wasm-smith` and `single-inst` as the module sources.
The intention behind this change is to have a *local* approach for fuzzing and
verifying programs generated by Winch and to have an initial implementation that
will allow us to eventually enable this change by default. Currently it's not
worth it to enable this change by default given all the filtering that needs to
happen to ensure that the generated modules are supported by Winch.
It's worth noting that the Wasm filtering code will be temporary, until Winch
reaches feature parity in terms of Wasm operators.
* Check build targets with the `winch` feature flag
* Rename fuzz target feature to `fuzz-winch`
This crate re-exports the `Backtrace` type at top-level from a nested
module. `Backtrace` in turn has `Frame` in its public API, which is not
re-exported anywhere. This is legal and external users can call methods
on `Frame`, but it doesn't appear in the rustdocs, making it
unnecessarily difficult to figure out how to use this API. Re-exporting
`Frame` fixes that, and also allows naming the type directly if needed.
OSS-Fuzz found a case where the `differential` fuzzer was failing and
the underlying cause was that Wasmtime was hitting an OOM while Wasmi
wasn't. This meant that the two modules were producing "different
results" since memories had differing lengths, but this isn't a failure
we're interested in. This commit updates the differential fuzzer to
discard the test case once the Wasmtime half reaches OOM.
* Make streams owned by request/response that they are tied to.
* Address comments, fix tests.
* Address comment.
* Update crates/wasi-http/src/streams_impl.rs
Co-authored-by: Pat Hickey <pat@moreproductive.org>
* Switch to BytesMut
---------
Co-authored-by: Pat Hickey <pat@moreproductive.org>
* Allow WASI to open directories without O_DIRECTORY
The `O_DIRECTORY` flag is a request that open should fail if the named
path is not a directory. Opening a path which turns out to be a
directory is not supposed to fail if this flag is not specified.
However, wasi-common required callers to use it when opening
directories.
With this PR, we always open the path the same way whether or not the
`O_DIRECTORY` flag is specified. However, after opening it, we `stat` it
to check whether it turned out to be a directory, and determine which
operations the file descriptor should support accordingly. In addition,
we explicitly check whether the precondition defined by `O_DIRECTORY` is
satisfied.
Closes#4947 and closes#4967, which were earlier attempts at fixing the
same issue, but which had race conditions.
prtest:full
* Add tests from #4967/#4947
This test was authored by Roman Volosatovs <rvolosatovs@riseup.net> as
part of #4947.
* Tests: Close FDs before trying to unlink files
On Windows, when opening a path which might be a directory using
`CreateFile`, cap-primitives also removes the `FILE_SHARE_DELETE` mode.
That means that if we implement WASI's `path_open` such that it always
uses `CreateFile` on Windows, for both files and directories, then
holding an open file handle prevents deletion of that file.
So I'm changing these test programs to make sure they've closed the
handle before trying to delete the file.
* Remove ModuleCompiledFunction
The same information can be retrieved using
ctx.compiled_code().unwrap().code_info().total_size
In addition for Module implementations that don't immediately compile the
given function there is no correct value that can be returned.
* Don't give anonymous functions and data objects an internal name
This internal name can conflict if a module is serialized and then
deserialized into another module. It also wasn't used by any of the
Module implementations anyway.
* Allow serializing all cranelift-module data structures
This allows a Module implementation to serialize it's internal state and
deserialize it in another compilation session. For example to implement
LTO or to load the module into cranelift-interpreter.
* Use expect
`poll_oneoff` uses `system_interface::ReadReady` to compute how many
bytes are ready to be read, which is part of the Preview1 `poll_oneoff`
API. This updates to system-interface 0.25.7 which has a fix to handle
special files such as /dev/urandom and /dev/null properly.
Fixes#6239.
This commit adds lowerings to the x64 backend for two more CLIF
instructions that currently require SSE 4.1. These lowerings are
inspired by LLVM's lowerings and avoid the use of SSE 4.1 instructions.
* riscv64: Remove unused code
* riscv64: Add vector types
* riscv64: Initial Vector ABI Load/Stores
* riscv64: Vector Loads/Stores
* riscv64: Fix `vsetvli` encoding error
* riscv64: Add SIMD `iadd` runtests
* riscv64: Rename `VecSew`
The SEW name is correct, but only for VType. We also use this type
in loads/stores as the Efective Element Width, so the name isn't
quite correct in that case.
* ci: Add V extension to RISC-V QEMU
* riscv64: Misc Cleanups
* riscv64: Check V extension in `load`/`store` for SIMD
* riscv64: Fix `sumop` doc comment
* cranelift: Fix comment typo
* riscv64: Add convert for VType and VecElementWidth
* riscv64: Remove VecElementWidth converter
This updates to rustix 0.37.13, which contains some features we can use to
implement more features in wasi-common for the wasi-sockets API. This also
pulls in several other updates to avoid having multiple versions of rustix.
This does introduce multiple versions of windows-sys, as the errno and tokio
crates are currently using 0.45 while rustix and other dependencies have
updated to 0.48; PRs updating these are already in flight so this will
hopefully be resolved soon.
It also includes cap-std 1.0.14, which disables the use of `openat2` and
`statx` on Android, fixing a bug where some Android devices crash the
process when those syscalls are executed.
* Fix default architecture for winch
This updates the `winch/codegen/build.rs` script to default to the
target architecture being compiled for as opposed to the host
architecture that's performing the compile.
Closes#6241
* Auto-enable other future architectures
This commit marks the loads of `*mut VMContext` and the callee function
pointer as `readonly` in the context of indirect function calls and
additionally calls to imported functions (which are indirect). Once a
`VMCallerCheckedAnyfunc` is initialized it's never modified so it should
be valid to mark these as readonly and if called in a loop should be
hoistable outside of the loop.
This test was not meaningfully executing, because wasi-common never
provides rights containing RIGHTS_PATH_FILESTAT_SET_SIZE - this flag is
not even defined in wasi-common/srd/dir.rs as one of the DirCaps flags.
Even when you get rid of that guard that skips the meat of the test,
path_open was being called with OFLAGS_TRUNC but without
RIGHTS_FD_WRITE, which boils down to an `open(2)` with OFLAGS_TRUNC set
and none of the access modes set, so it will always fail with EINVAL.
So, it doesn't look like this test ever would have meaningfully passed,
even in pre-wiggle-rewrite version of wasi-common it landed in back in
late 2019. Late 2019! before the pandemic! our eyes were so full of
stars and dreams of the future!
The behavior we really care about for truncation are taken care of
by the fd_filestat_set test, which shows fd_filestat_set_size works
correctly, and the file_truncation test, which shows that opening
with OFLAGS_TRUNC will truncate the file.
* Add support for binary/octal literals to ISLE
In a number of x64-changes recently some u8 immediates are interpreted
as four bit-packed 2-bit numbers and I have a tough time going between
hex and these bit-packed numbers. I've been writing `0xAA == 0b...` in
comments to indicate the intent but I figured it'd be a bit clearer if
the binary literal was accepted directly!
This is a minor update to the ISLE lexer to allow for binary `0b00...`
and octal `0o00...` literals in the same manner as hex literals. Some
comments in the x64 backend are then removed to use the binary literal
syntax directly.
* Update ISLE reference for octal/binary
* Update ISLE tests for octal/binary
This method returns a Hash, the output of which can be used to index
precompiled binaries from one Engine instance that can be deserialized
by another Engine instance.
Wasmtime disallows guests from using `path_symlink` to create
absolute-path symlinks, as they could confuse other code into
accessing resources on the host that the guest otherwise doesn't
have access to.
This patch adds a test for this behavior.
* x64: Add non-SSE4.1 lowerings of ceil/trunc/floor/nearest
This commit adds lowerings that work with SSE2 for CLIF `ceil`, `trunc`,
`floor`, and `nearest` instructions over vectors. To get these working
`insertlane` for float vectors was also implemented for non-SSE4.1
instructions as well.
Note that the goal of these lowerings is not speed but rather "it
works", so the decompose-to-call-libcalls logic for vector is probably
horrendously slow but should at least be correct.
* Skip new tests on riscv64
* Update cranelift/codegen/src/isa/x64/inst.isle
Co-authored-by: Andrew Brown <andrew.brown@intel.com>
---------
Co-authored-by: Andrew Brown <andrew.brown@intel.com>
* Gracefully handle ENOTSUP for `memfd_create`
Looks like in #6229 older kernels may not support memfd in which case we
can fall back to manual memory initialization gracefully.
Closes#6229
* Fix tests
* wasi-common: remove allocate from WasiFile trait, always fail with NOTSUP
This operation from cloudabi is linux-specific, isn't even
supported across all linux filesystems, and has no support on macos
or windows. Rather than ship spotty support, it has been removed
from preview 2, and we are no longer supporting it in preview 1 as
well.
The preview 1 implementation will still check if fd is a file, and has
rights, just to reject those cases with the errors expected.
* wasi-tests: expect fd_allocate to always fail now.
rewrite the file_allocate test to just check for failure.
remove use of fd_allocate from fd_advise test, and remove test
configuration setting used for excluding use of fd_allocate on macos and
windows.
* Optimize sign extension via shifts
This commit adds egraph optimization patterns for left-shifting a value
and then right-shifting it as a form of sign extending its lower bits.
This matches the behavior of the WebAssembly `i32.extend8_s`
instruction, for example. Note that the lowering of that WebAssembly
instruction does not use shifts, but historical versions of LLVM that
didn't support the instruction, or versions with the instruction
disabled, will use shifts instead.
A second rule for reduction-of-extend being the same as the original
value was added to keep an existing shift-related test passing as well.
* Add reference assemblies for new opts
* x64: Change `use_sse41` to a constructor
This refactors the existing `use_sse41` extractor to instead be a
`constructor` to use with `if-let`.
* x64: Gate the `pblendw` instruction on SSE4.1 being enabled
This specialization of `shuffle` isn't a base case so adding an `if-let`
here should be sufficient for gating this instruction properly on
enabled CPU features.
* x64: Gate `pmuldq` lowerings on SSE 4.1
The specialized rules using these instructions can fall back to the
standard lowerings for non-SSE 4.1 instructions.
While bringing in no major updates for Wasmtime I've taken this
opportunity to list myself for `cargo vet` with wildcard audits of this
family of crates. That means I shouldn't need to further add any more
entries in the future for updating these crates and additionally any
other organizations using these audits will automatically be able to
have audits for version that I publish.
While here I also ran `cargo vet prune` which was able to remove a
number of our exemptions.
This commit improves ABI support in Winch's trampolines mainly by:
* Adding support for the `fastcall` calling convention.
* By storing/restoring callee-saved registers.
One of the explicit goals of this change is to make tests available in the x86_64 target
as a whole and remove the need exclude the windows target.
This commit also introduces a `CallingConvention` enum, to better
reflect the subset of calling conventions that are supported by Winch.
Cranelift only has one instruction SIMD which depends on SSE4.2 so this
commit adds a lowering rule for `pcmpgtq` which doesn't use SSE4.2 and
enables lowering the baseline requirement for SIMD support from SSE4.2
to SSE4.1.
The `has_sse42` setting is no longer enabled by default for Cranelift.
Additionally `enable_simd` no longer requires `has_sse42` on x64.
Finally the fuzz-generator for Wasmtime codegen settings now enables
flipping the `has_sse42` setting instead of unconditionally setting it
to `true`.
The specific lowering for `pcmpgtq` is copied from LLVM's lowering of
this instruction.
