This commit does a bit of everything: refactors bits here and there,
fixes a bug discovered in another #701, and combines all structs that
we used in `yanix` and `wasi-common` crates to represent file types
on *nix into one struct, `yanix::file::FileType`.
Up until now, in `yanix`, we've had two separate structs used to
represent file types on the host: `yanix::dir::FileType` and
`yanix::file::SFlags` (well, not quite, but that was its main use).
They both were used in different context (the former when parsing
`dirent` struct, and the latter when parsing `stat` struct), they
were C-compatible (as far as their representation goes), and as it
turns out, they shared possible enumeration values. This commit
combines them both into an idiomatic Rust enum with the caveat that
it is now *not* C-compatible, however, I couldn't find a single use
where that would actually matter, and even if it does in the future,
we can simply add appropriate impl methods.
The combine `yanix::file::FileType` struct can be constructed in two
ways: 1) either from `stat.st_mode` value (and while we're here,
now it's done correctly according to POSIX which fixes the bug mentioned
in VFS impl PR #701), or 2) from `dirent.d_type` value. Also, since we now
have one struct for representing both contexts, this cleans up nicely
a lot of duplicated code in `host` module.
* witx tagged unions: updates to wig to use new semantics
* wig: emit a `#variant: ()` union variant for empty variants
* wasi-common: translate to use tagged unions
* update to flattened layout of event struct
* wig: generate layout tests, and delete bindgen ones
the bindgen tests became out-of-date with the latest changes to the
representation of unions, and the re-jiggering of various struct
definitions that went along with it.
* wasi: point at master with tagged union PR merged
* fix event struct repr on windows
* Log str repr of WASI errno at trace level
This commit refactors `Error` enum, and adds logging of the WASI
errno string representation at the trace level. Now, when tracing
WASI syscalls, we will be greeted with a nicely formatted errno
value after each syscall:
```
path_open(...)
| *fd=5
| errno=ESUCCESS
```
This commit gets rid of `errno_from_nix`, `errno_from_win` and
`errno_from_host` helper fns in favour of direct `From` implementations
for the relevant types such as `yanix::Errno` and `winx::winerror::WinError`.
`errno_from_host` is replaced by a trait `FromRawOsError`.
* Back port changes to snapshot0
* Fix indentation in logs
This commit implements `fd_fdstat_set_flags` for Windows.
Additionally, it fixes a problem where `O_APPEND` was not working correctly
because `GENERIC_WRITE` was always being set; as a result, `FILE_WRITE_DATA`
could not be removed from the permission set to properly enable append-only
mode.
It also treats `O_TRUNC` with `O_APPEND` as an invalid argument error. This is
because Windows cannot support these two flags together. To support `O_TRUNC`,
the `GENERIC_WRITE` bit must be set for the file access flags. Setting this
bit will cause `FILE_WRITE_DATA` to be set, which will not properly treat the
file as append-only (it requires `FILE_APPEND_DATA` without `FILE_WRITE_DATA`).
* Clean up fd_filestat_get implementation
This commit does 4 things:
* Adds `yanix::file::fstat`, a wrapper around `libc::fstat`.
* It essentially reverts 89fbde2 for Unix hosts -- in other words,
it brings back the use of `fstat` to obtain `libc::stat` from a
file descriptor, rather than relying on `std::fs::Metadata`. This
way, we reuse `host_impl::filestat_from_nix` in
`hostcalls_impl::fd_filestat_get` implementation rather than
unnecessarily duplicate code for converting filestats into
`__wasi_filestat_t`.
* Moves `crate::helpers::systemtime_to_timestamp` to Windows `host_impl`
module. It does the same thing with helpers which assist in converting
`std::fs::Metadata` into `__wasi_filestat_t`. This should retain symmetry
between *nix and Windows impls.
* Makes timestamp conversions in `host_impl::filestat_from_nix` fallible.
* Backport changes to snapshot0
* Signal no overflow with `from` rather than `as` cast
* Correctly handle possibly misaligned pointers in readdir
This reapplies #615, which was inadvertently reverted.
* Tidy up unneeded `self::` qualifiers.
* Make Dir's contents private.
Also remove the `unsafe` from `impl_iter`. With `Dir`'s field being
private, we can rely on the pointer being only what we've assigned to
it.
* Make `poll`'s timeout argument a `libc::c_int`.
This clarifies that there are no subsequent conversions before calling the
underlying libc API.
* Use clock_gettime instead of clock_getres to get the time.
* Mark FileType::from_raw as safe.
It handles unknown values, so it can be marked safe.
Only very recently in #700 did we actually start running wasi tests
again (they weren't running by accident). Just before that landed we
also landed #688 which had some refactorings. Unfortunately #688 had a
minor issue in it which wasn't caught because tests weren't run. This
means that the bug in #688 slipped in and is now being caught by #700
now that both are landed on master.
This commit fixes the small issue introduced and should get our CI green
again!
* Compile wasi-common to Emscripten
This commit enables cross-compiling of `wasi-common` to Emscripten. To achieve
this, this commit does quite a bit reshuffling in the existing codebase. Namely,
* rename `linux` modules in `wasi-common` and `yanix` to `linux_like` -- this is
needed so that we can separate out logic specific to Linux and Emscripten out
* tweak `dir` module in `yanix` to support Emscripten -- in particular, the main
change involves `SeekLoc::from_raw` which has to be now host-specific, and is now
fallible
* tweak `filetime` so that in Emscripten we never check for existence of `utimensat`
at runtime since we are guaranteed for it to exist by design
* since `utimes` and `futimes` are not present in Emscripten, move them into a separate
module, `utimesat`, and tag it cfg-non-emscripten only
* finally, `to_timespec` is now fallible since on Emscripten we have to cast number of
seconds, `FileTime::seconds` from `i64` to `libc::c_long` which resolves to `i32`
unlike on other nixes
* Fix macos build
* Verify wasi-common compiles to Emscripten
This commit adds `emscripten` job to Github Actions which installs
`wasm32-unknown-emscripten` target, and builds `wasi-common` crate.
* Use #[path] to cherry-pick mods for Emscripten
This commit effectively reverses the reorg introduced in 145f4a5
in that it ditches `linux_like` mod for separate mods `linux` and
`emscripten` which are now on the same crate level, and instead,
pulls in common bits from `linux` using the `#[path = ..]` proc
macro.
* Add yanix crate
This commit adds `yanix` crate as a Unix dependency for `wasi-common`.
`yanix` stands for Yet Another Nix crate and is exactly what the name
suggests: a crate in the spirit of the `nix` crate, but which takes a different
approach, using lower-level interfaces with less abstraction, so that it fits
better with its main use case, implementation of WASI syscalls.
* Replace nix with yanix crate
Having introduced `yanix` crate as an in-house replacement for the
`nix` crate, this commit makes the necessary changes to `wasi-common`
to depend _only_ on `yanix` crate.
* Address review comments
* make `fd_dup` unsafe
* rename `get_fd` to `get_fd_flags`, etc.
* reuse `io::Error::last_os_error()` to get the last errno value
* Address more comments
* make all `fcntl` fns unsafe
* adjust `wasi-common` impl appropriately
* Make all fns operating on RawFd unsafe
* Fix linux build
* Address more comments
* Support fd_fdstat_get on stdin/stdout/stderr.
Add a routine for obtaining an `OsFile` containing a file descriptor for
stdin/stdout/stderr so that we can do fd_fdstat_get on them.
* Add a testcase for fd_fdstat_get etc. on stdin etc.
* Don't dup file descriptors in fd_renumber.
* Fix compilation on macOS
* Rename OsFile to OsHandle
This commits renames `OsFile` to `OsHandle` which seems to make
more sense semantically as it is permitted to hold a valid OS handle
to OS entities other than simply file/dir (e.g., socket, stream, etc.).
As such, this commit also renames methods on `Descriptor` struct
from `as_actual_file` to `as_file` as this in reality does pertain
ops on FS entities such as files/dirs, and `as_file` to `as_os_handle`
as in this case it can be anything, from file, through a socket, to
a stream.
* Fix compilation on Linux
* Introduce `OsHandleRef` for borrowing OS resources.
To prevent a `ManuallyDrop<OsHandleRef>` from outliving the resource it
holds on to, create an `OsHandleRef` class parameterized on the lifetime
of the `Descriptor`.
* Fix scoping to pub-priv and backport to snapshot_0
This commit fully implements `__wasi_fd_fdstat_get` on Windows so that
the descriptor flags can be determined.
It does this by calling into `NtQueryInformationFile` (safe to call from
user mode) to get the open mode and access of the underlying OS handle.
`NtQueryInformationFile` isn't included in the `winapi` crate, so it is
manually being linked against.
This commit also fixes several bugs on Windows:
* Ignore `__WASI_FDFLAG_NONBLOCK` by not setting `FILE_FLAG_OVERLAPPED`
on file handles (the POSIX behavior for `O_NONBLOCK` on files).
* Use `FILE_FLAG_WRITE_THROUGH` for the `__WASI_FDFLAG_?SYNC` flags.
* `__WASI_FDFLAG_APPEND` should disallow `FILE_WRITE_DATA` access to
force append-only on write operations.
* Use `GENERIC_READ` and `GENERIC_WRITE` access flags. The
latter is required when opening a file for truncation.
* Unify fd_readdir impl between *nixes
This commit unifies the implementation of `fd_readdir` between Linux
and BSD hosts. In particular, it re-uses the `Dirent`, `Entry`, and
`Dir` (among others) building blocks introduced recently when
`fd_readdir` was being implemented on Windows.
Notable changes:
* on BSD, wraps `readdir` syscall in an `Iterator` of the mutex-locked
`Dir` struct
* on BSD, removes `DirStream` struct from `OsFile`; `OsFile` now holds a
mutex to `Dir`
* makes `Dir` iterators implementation specific (Linux has its own,
and so does BSD)
* Lock mutex once only; explain dir in OsFile
* Add more comments
* Add support for wasi_snapshot_preview1.
This adds support for the new ABI, while preserving compatibility
support for the old ABI.
* Fix compilation on platforms where nlink_t isn't 64-bit.
* rustfmt
* Fix Windows build errors.
* Dynamically load utimensat if exists on the host
This commit introduces a change to file time management for *nix based
hosts in that it firstly tries to load `utimensat` symbol, and if it
doesn't exist, then falls back to `utimes` instead. This change is
borrowing very heavily from [filetime] crate, however, it introduces a
couple of helpers and methods specific to WASI use case (or more
generally, to a use case which requires modifying times of entities
specified by a pair `(DirFD, RelativePath)` rather than the typical
file time specification based only absolute path or raw file descriptor
as is the case with [filetime] crate. The trick here is, that on kernels
which do not have `utimensat` symbol, this implementation emulates this
behaviour by a combination of `openat` and `utimes`.
This commit also is meant to address #516.
[filetime]: https://github.com/alexcrichton/filetime
* Fix symlink NOFOLLOW flag setting
* Add docs and specify UTIME_NOW/OMIT on Linux
Previously, we relied on [libc] crate for `UTIME_NOW` and `UTIME_OMIT`
constants on Linux. However, following the convention assumed in
[filetime] crate, this is now changed to directly specified by us
in our crate.
[libc]: https://github.com/rust-lang/libc
[filetime]: https://github.com/alexcrichton/filetime
* Refactor UTIME_NOW/OMIT for BSD
* Address final discussion points
