* Clean up fd_filestat_get implementation
This commit does 4 things:
* Adds `yanix::file::fstat`, a wrapper around `libc::fstat`.
* It essentially reverts 89fbde2 for Unix hosts -- in other words,
it brings back the use of `fstat` to obtain `libc::stat` from a
file descriptor, rather than relying on `std::fs::Metadata`. This
way, we reuse `host_impl::filestat_from_nix` in
`hostcalls_impl::fd_filestat_get` implementation rather than
unnecessarily duplicate code for converting filestats into
`__wasi_filestat_t`.
* Moves `crate::helpers::systemtime_to_timestamp` to Windows `host_impl`
module. It does the same thing with helpers which assist in converting
`std::fs::Metadata` into `__wasi_filestat_t`. This should retain symmetry
between *nix and Windows impls.
* Makes timestamp conversions in `host_impl::filestat_from_nix` fallible.
* Backport changes to snapshot0
* Signal no overflow with `from` rather than `as` cast
* Use thiserror proc macros for auto From impls
This commit refactors `wasi_common::error::Error` by using `#[from]`
proc macro to autoderive `From` for wrapped errors.
* Back port changes to snapshot0
* Auto impl Display for WasiError
* Fix stack overflow when auto generating Display for WasiError
* Initial support for securing tty I/O.
* Update the tests.
* Fix warnings
* Update crates/wasi-common/src/fdentry.rs
Co-Authored-By: Jakub Konka <jakub.konka@golem.network>
* Properly sandbox stderr.
* Document why the scratch buffer is 4 elements long.
* Update crates/wasi-common/src/sandboxed_tty_writer.rs
Co-Authored-By: Jakub Konka <jakub.konka@golem.network>
* Update crates/wasi-common/src/sandboxed_tty_writer.rs
Co-Authored-By: Jakub Konka <jakub.konka@golem.network>
* Add comments explaining how we report the number of bytes written.
* Always sanitize stderr.
* Port the changes to the snapshot_0 directory.
* Fix snapshot_0 compilation error.
* Replace the scratch buffer with a temporary buffer.
* Update crates/wasi-common/src/sandboxed_tty_writer.rs
Co-Authored-By: bjorn3 <bjorn3@users.noreply.github.com>
* Format with latest stable rustfmt.
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
Co-authored-by: bjorn3 <bjorn3@users.noreply.github.com>
This commit uses the `*.witx` files describing the current wasi API to
reduce the boilerplate used to define implementations in the
`wasmtime-wasi` crate. Eventually I'd like to remove lots of boilerplate
in the `wasi-common` crate too, but this should at least be a good start!
The boilerplate removed here is:
* No need to list each function to add it to the
`wasmtime_runtime::Module` being created
* No need to list the signature of the function in a separate
`syscalls.rs` file.
Instead the `*.witx` file is processed in a single-use macro inside the
`wasmtime-wasi` crate. This macro uses the signatures known from
`*.witx` to automatically register with the right type in the wasm
module as well as define a wrapper that the wasm module will call into.
Functionally this is all the same as before, it's just defined in a
different way now!
The shim generated by this macro which wasmtime calls into only uses
`i32`/`i64`/etc wasm types, and it internally uses `as` casts to convert
to the right wasi types when delegating into the `wasi-common` crate.
One change was necessary to get this implemented, however. The functions
in `wasi-common` sometimes took `WasiCtx` and sometimes took a slice of
memory. After this PR they uniformly all require both `WasiCtx` and
memory so the wrappers can be auto-generated. The arguments are ignored
if they weren't previously required.
* Correctly handle possibly misaligned pointers in readdir
This reapplies #615, which was inadvertently reverted.
* Tidy up unneeded `self::` qualifiers.
* Make Dir's contents private.
Also remove the `unsafe` from `impl_iter`. With `Dir`'s field being
private, we can rely on the pointer being only what we've assigned to
it.
* Make `poll`'s timeout argument a `libc::c_int`.
This clarifies that there are no subsequent conversions before calling the
underlying libc API.
* Use clock_gettime instead of clock_getres to get the time.
* Mark FileType::from_raw as safe.
It handles unknown values, so it can be marked safe.
Only very recently in #700 did we actually start running wasi tests
again (they weren't running by accident). Just before that landed we
also landed #688 which had some refactorings. Unfortunately #688 had a
minor issue in it which wasn't caught because tests weren't run. This
means that the bug in #688 slipped in and is now being caught by #700
now that both are landed on master.
This commit fixes the small issue introduced and should get our CI green
again!
* Compile wasi-common to Emscripten
This commit enables cross-compiling of `wasi-common` to Emscripten. To achieve
this, this commit does quite a bit reshuffling in the existing codebase. Namely,
* rename `linux` modules in `wasi-common` and `yanix` to `linux_like` -- this is
needed so that we can separate out logic specific to Linux and Emscripten out
* tweak `dir` module in `yanix` to support Emscripten -- in particular, the main
change involves `SeekLoc::from_raw` which has to be now host-specific, and is now
fallible
* tweak `filetime` so that in Emscripten we never check for existence of `utimensat`
at runtime since we are guaranteed for it to exist by design
* since `utimes` and `futimes` are not present in Emscripten, move them into a separate
module, `utimesat`, and tag it cfg-non-emscripten only
* finally, `to_timespec` is now fallible since on Emscripten we have to cast number of
seconds, `FileTime::seconds` from `i64` to `libc::c_long` which resolves to `i32`
unlike on other nixes
* Fix macos build
* Verify wasi-common compiles to Emscripten
This commit adds `emscripten` job to Github Actions which installs
`wasm32-unknown-emscripten` target, and builds `wasi-common` crate.
* Use #[path] to cherry-pick mods for Emscripten
This commit effectively reverses the reorg introduced in 145f4a5
in that it ditches `linux_like` mod for separate mods `linux` and
`emscripten` which are now on the same crate level, and instead,
pulls in common bits from `linux` using the `#[path = ..]` proc
macro.
* Add yanix crate
This commit adds `yanix` crate as a Unix dependency for `wasi-common`.
`yanix` stands for Yet Another Nix crate and is exactly what the name
suggests: a crate in the spirit of the `nix` crate, but which takes a different
approach, using lower-level interfaces with less abstraction, so that it fits
better with its main use case, implementation of WASI syscalls.
* Replace nix with yanix crate
Having introduced `yanix` crate as an in-house replacement for the
`nix` crate, this commit makes the necessary changes to `wasi-common`
to depend _only_ on `yanix` crate.
* Address review comments
* make `fd_dup` unsafe
* rename `get_fd` to `get_fd_flags`, etc.
* reuse `io::Error::last_os_error()` to get the last errno value
* Address more comments
* make all `fcntl` fns unsafe
* adjust `wasi-common` impl appropriately
* Make all fns operating on RawFd unsafe
* Fix linux build
* Address more comments
While we are waiting for the Rust toolchain to use the new ABI,
I thought it might be useful to sync `snapshot_0` with the latest
code in `wasi-common` "upstream". This mainly includes the latest
refactoring effort to unify the machinery for `fd_readdir` between
Linux, Windows and BSD.
* Support fd_fdstat_get on stdin/stdout/stderr.
Add a routine for obtaining an `OsFile` containing a file descriptor for
stdin/stdout/stderr so that we can do fd_fdstat_get on them.
* Add a testcase for fd_fdstat_get etc. on stdin etc.
* Don't dup file descriptors in fd_renumber.
* Fix compilation on macOS
* Rename OsFile to OsHandle
This commits renames `OsFile` to `OsHandle` which seems to make
more sense semantically as it is permitted to hold a valid OS handle
to OS entities other than simply file/dir (e.g., socket, stream, etc.).
As such, this commit also renames methods on `Descriptor` struct
from `as_actual_file` to `as_file` as this in reality does pertain
ops on FS entities such as files/dirs, and `as_file` to `as_os_handle`
as in this case it can be anything, from file, through a socket, to
a stream.
* Fix compilation on Linux
* Introduce `OsHandleRef` for borrowing OS resources.
To prevent a `ManuallyDrop<OsHandleRef>` from outliving the resource it
holds on to, create an `OsHandleRef` class parameterized on the lifetime
of the `Descriptor`.
* Fix scoping to pub-priv and backport to snapshot_0
This commit fully implements `__wasi_fd_fdstat_get` on Windows so that
the descriptor flags can be determined.
It does this by calling into `NtQueryInformationFile` (safe to call from
user mode) to get the open mode and access of the underlying OS handle.
`NtQueryInformationFile` isn't included in the `winapi` crate, so it is
manually being linked against.
This commit also fixes several bugs on Windows:
* Ignore `__WASI_FDFLAG_NONBLOCK` by not setting `FILE_FLAG_OVERLAPPED`
on file handles (the POSIX behavior for `O_NONBLOCK` on files).
* Use `FILE_FLAG_WRITE_THROUGH` for the `__WASI_FDFLAG_?SYNC` flags.
* `__WASI_FDFLAG_APPEND` should disallow `FILE_WRITE_DATA` access to
force append-only on write operations.
* Use `GENERIC_READ` and `GENERIC_WRITE` access flags. The
latter is required when opening a file for truncation.
* Unify fd_readdir impl between *nixes
This commit unifies the implementation of `fd_readdir` between Linux
and BSD hosts. In particular, it re-uses the `Dirent`, `Entry`, and
`Dir` (among others) building blocks introduced recently when
`fd_readdir` was being implemented on Windows.
Notable changes:
* on BSD, wraps `readdir` syscall in an `Iterator` of the mutex-locked
`Dir` struct
* on BSD, removes `DirStream` struct from `OsFile`; `OsFile` now holds a
mutex to `Dir`
* makes `Dir` iterators implementation specific (Linux has its own,
and so does BSD)
* Lock mutex once only; explain dir in OsFile
* Add more comments
* Fix feature-gating of test-programs
This commit fixes bugs in enabling feature-gating of `test-programs`
which was introduced in #600. It turns out, #600 accidentally
disabled `test-programs` from ever running, and this commit fixes
that.
* Fix the CI
* Add support for wasi_snapshot_preview1.
This adds support for the new ABI, while preserving compatibility
support for the old ABI.
* Fix compilation on platforms where nlink_t isn't 64-bit.
* rustfmt
* Fix Windows build errors.
* Add test for dangling file/dir handles
This commit adds a test for dangling file/dir handles. The logic is
quite simple: we first create a resource (file or dir), get a WASI file
descriptor to it, remove the resource without closing the FD, and then
try to re-create it.
* Disable on Windows for now
* Fix fuzz target compilation.
* Bump version to 0.7.0
* Temporarily disable fuzz tests
Temporarily disable fuzz tests until https://github.com/bytecodealliance/cranelift/issues/1216 is resolved.
* Fix publish-all.sh to not modify the witx crate.
* Remove the "publish = false" attribute from Lightbeam.
* Add a README.md for wasmtime-interface-types.
* Remove the "rust" category.
This fixes the following warning:
warning: the following are not valid category slugs and were ignored: rust. Please see https://crates.io/category_slugs for the list of all category slugs.
* Mark wasmtime-cli as "publish = false".
* Sort the publishing rules in topological order.
Also, publish nightly-only crates with cargo +nightly.
This commit fixes rights check for `fd_pread` and `fd_pwrite` to be
conformant with the WASI spec. In the spec, it is clearly stated that
the right to invoke `__wasi_fd_pread()` requires a combination of
`__WASI_RIGHT_FD_READ` with `__WASI_RIGHT_FD_SEEK`, and similarly for
`__wasi_fd_pwrite()` the combination is `__WASI_RIGHT_FD_WRITE` with
`__WASI_RIGHT_FD_SEEK`. Relevant link to the spec: [__wasi_rights_t].
[__wasi_rights_t]: https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs/wasi_unstable_preview1.md#__wasi_rights_t-uint64_t-bitfield
* Reorganize wasi-misc-tests.
Move wasi-misc-tests out of wasi-common, to break a dependency cycle;
previously, wasmtime-* depended on wasi-common, but wasi-common
dev-dependended on wasmtime-*.
Now, wasi-common no longer dev-depends on wasmtime-*; instead, the
tests are in their own crate which depends on wasi-common and on
wasmtime-*.
Also, rename wasi-misc-tests to wasi-tests for simplicity.
This also removes the "wasm_tests" feature; it's replaced by the
"test-programs" feature.
* Update the CI script to use the new feature name.
* Update the CI script to use the new feature name in one more place.
* Change a `write!` to a `writeln!`.
This commit is an attempt to reduce the number of crates necessary to
link to when using `wasmtime::Config` in "default mode" or with only one
or two tweaks. The change moves to a builder-style pattern for `Config`
to only require importing crates as necessary if you configure a
particular setting. This then also propagates that change to `Context`
as well by taking a `Config` instead of requiring that all arguments are
passed alone.
For some weird reason (probably when migrating the codebase from
`wasi-common` repo to `wasmtime`), these did not get enabled for the
Windows platform.
This commit adds a relatively complete test case for the `path_link`
syscall. This commit should serve as some prep work for implementing
`path_link` on Windows (which will follow in a subsequent PR).
* Allow using WASI APIs in the Python extension
This commit adds support to the Python extension to load the WASI
implementation when a WASI module is seen allowing Python to load
WebAssembly modules that use WASI. This is pretty primitive right now
because there's no way to configure the environment/args/preopens/etc,
but it's hoped to be at least a start!
* rustfmt
* Refactor checks for the wasi module name
* Move the check into `wasmtime-wasi` itself
* Make it conservative for now and match anything that says `wasi*`
* Leave a `FIXME` for improving this later on
* Enable missing feature of winapi for `winx`
* Dynamically load utimensat if exists on the host
This commit introduces a change to file time management for *nix based
hosts in that it firstly tries to load `utimensat` symbol, and if it
doesn't exist, then falls back to `utimes` instead. This change is
borrowing very heavily from [filetime] crate, however, it introduces a
couple of helpers and methods specific to WASI use case (or more
generally, to a use case which requires modifying times of entities
specified by a pair `(DirFD, RelativePath)` rather than the typical
file time specification based only absolute path or raw file descriptor
as is the case with [filetime] crate. The trick here is, that on kernels
which do not have `utimensat` symbol, this implementation emulates this
behaviour by a combination of `openat` and `utimes`.
This commit also is meant to address #516.
[filetime]: https://github.com/alexcrichton/filetime
* Fix symlink NOFOLLOW flag setting
* Add docs and specify UTIME_NOW/OMIT on Linux
Previously, we relied on [libc] crate for `UTIME_NOW` and `UTIME_OMIT`
constants on Linux. However, following the convention assumed in
[filetime] crate, this is now changed to directly specified by us
in our crate.
[libc]: https://github.com/rust-lang/libc
[filetime]: https://github.com/alexcrichton/filetime
* Refactor UTIME_NOW/OMIT for BSD
* Address final discussion points
* General Cargo.toml cleanup.
- Remove travis-ci attributes.
- Remove "experimental" badges from actively-developed crates.
- Reflow some long lines.
- Use dependency features consistently.
- Add readme attributes
* Update WASI to the latest trunk.
This notably adds a .gitignore file for the WASI directory.
