cbf7cbfa39f3090fd10f6660863980dfbfad72c3
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Jakub Konka
|
cbf7cbfa39 |
Introduce strongly-typed system primitives (#1561)
* Introduce strongly-typed system primitives This commit does a lot of reshuffling and even some more. It introduces strongly-typed system primitives which are: `OsFile`, `OsDir`, `Stdio`, and `OsOther`. Those primitives are separate structs now, each implementing a subset of `Handle` methods, rather than all being an enumeration of some supertype such as `OsHandle`. To summarise the structs: * `OsFile` represents a regular file, and implements fd-ops of `Handle` trait * `OsDir` represents a directory, and primarily implements path-ops, plus `readdir` and some common fd-ops such as `fdstat`, etc. * `Stdio` represents a stdio handle, and implements a subset of fd-ops such as `fdstat` _and_ `read_` and `write_vectored` calls * `OsOther` currently represents anything else and implements a set similar to that implemented by `Stdio` This commit is effectively an experiment and an excercise into better understanding what's going on for each OS resource/type under-the-hood. It's meant to give us some intuition in order to move on with the idea of having strongly-typed handles in WASI both in the syscall impl as well as at the libc level. Some more minor changes include making `OsHandle` represent an OS-specific wrapper for a raw OS handle (Unix fd or Windows handle). Also, since `OsDir` is tricky across OSes, we also have a supertype of `OsHandle` called `OsDirHandle` which may store a `DIR*` stream pointer (mainly BSD). Last but not least, the `Filetype` and `Rights` are now computed when the resource is created, rather than every time we call `Handle::get_file_type` and `Handle::get_rights`. Finally, in order to facilitate the latter, I've converted `EntryRights` into `HandleRights` and pushed them into each `Handle` implementor. * Do not adjust rights on Stdio * Clean up testing for TTY and escaping writes * Implement AsFile for dyn Handle This cleans up a lot of repeating boilerplate code todo with dynamic dispatch. * Delegate definition of OsDir to OS-specific modules Delegates defining `OsDir` struct to OS-specific modules (BSD, Linux, Emscripten, Windows). This way, `OsDir` can safely re-use `OsHandle` for raw OS handle storage, and can store some aux data such as an initialized stream ptr in case of BSD. As a result, we can safely get rid of `OsDirHandle` which IMHO was causing unnecessary noise and overcomplicating the design. On the other hand, delegating definition of `OsDir` to OS-specific modules isn't super clean in and of itself either. Perhaps there's a better way of handling this? * Check if filetype of OS handle matches WASI filetype when creating It seems prudent to check if the passed in `File` instance is of type matching that of the requested WASI filetype. In other words, we'd like to avoid situations where `OsFile` is created from a pipe. * Make AsFile fallible Return `EBADF` in `AsFile` in case a `Handle` cannot be made into a `std::fs::File`. * Remove unnecessary as_file conversion * Remove unnecessary check for TTY for Stdio handle type * Fix incorrect stdio ctors on Unix * Split Stdio into three separate types: Stdin, Stdout, Stderr * Rename PendingEntry::File to PendingEntry::OsHandle to avoid confusion * Rename OsHandle to RawOsHandle Also, since `RawOsHandle` on *nix doesn't need interior mutability wrt the inner raw file descriptor, we can safely swap the `RawFd` for `File` instance. * Add docs explaining what OsOther is * Allow for stdio to be non-character-device (e.g., piped) * Return error on bad preopen rather than panic |
||
|
Jakub Konka
|
de919382b3 |
Make Handle a trait required for any WASI-compatible handle (#1443)
* Make Handle a trait required for any WASI-compatible handle
OK, so this PR is a bit of an experiment that came about somewhat itself
when I was looking at refactoring use of `Rc<RefCell<Descriptor>>` inside
`Entry` struct. I've noticed that since we've placed `VirtualFile` on the
same level as `OsHandle` and `Stdin` etc., we've ended up necessiitating
checks for different combinations such as "is a real OS resource being mixed
up with a virtual resource?", and if that was the case, we'd panic since
this was clearly not allowed (e.g., symlinking, or worse renaming).
Therefore, it seemed natural for virtual file to be on the same level
as _any_ OS handle (regardless of whether it's an actual file, socket,
or stdio handle). In other words, we should ideally envision the following
hierarchy:
```
\-- OsHandle \-- OsFile
-- Stdio
\-- Virtual
```
This way, we can deal with the mix up at a level above which cleans up
our logic significantly.
On the other hand, when looking through the `virtfs`, the trait approach
to some type that's a valid `Handle` grew on me, and I think this
is the way to go. And this is what this PR is proposing, a trait
`Handle` which features enough functionality to make both virtual and
OS ops to work. Now, inside `Entry` we can safely store something like
`Rc<dyn Handle>` where `Handle` can downcast to either `VirtualFile` or
`VirtualDir`, or `OsHandle` if its an actual OS resource. Note that
I've left `Handle` as one massive trait, but I reckon we could split
it up into several smaller traits, each dealing with some bit of WASI
functionality. I'm hoping this would perhaps make it easier to figure
out polyfilling between snapshots and the new upcoming ephemeral
snapshot since a lot of boilerplate functionality is now done as part
of the `Handle` trait implementation.
Next, I've redone the original `OsHandle` to be an `OsFile` which
now stores a raw descriptor/handle (`RawFd`/`RawHandle`) inside a
`Cell` so that we can handle interior mutability in an easy (read,
non-panicky) way. In order not to lose the perks of derefercing to
`std::fs::File`, I've added a convenience trait `AsFile` which
will take `OsFile` by reference (or the stdio handles) and create
a non-owned `ManuallyDrop<File>` resource which can be passed around
and acted upon the way we'd normally do on `&File`. This change of
course implies that we now have to worry about properly closing all
OS resources stored as part of `OsFile`, thus this type now implements
`Drop` trait which essentially speaking moves the raw descriptor/handle
into a `File` and drops it.
Finally, I've redone setting time info on relative paths on *nix using
the same approach as advocated in the virtual fs. Namely, we do an
`openat` followed by `filestat_set_times` on the obtained descriptor.
This effectively removes the need for custom `filetime` module in
`yanix`. However, this does probably incur additional cost of at least
one additional syscall, and I haven't checked whether this approach
performs as expected on platforms such as NixOS which as far as I remember
had some weirdness todo with linking `utimensat` symbols, etc. Still,
this change is worth considering given that the implementation of
`path_filestat_set_times` cleans up a lot, albeit with some additional
cost.
* Fix tests on Windows
* Address comments plus minor consistency cleanup
* Address comments
* Fix formatting
|
||
|
Jakub Konka
|
32595faba5 |
It's wiggle time! (#1202)
* Use wiggle in place of wig in wasi-common This is a rather massive commit that introduces `wiggle` into the picture. We still use `wig`'s macro in `old` snapshot and to generate `wasmtime-wasi` glue, but everything else is now autogenerated by `wiggle`. In summary, thanks to `wiggle`, we no longer need to worry about serialising and deserialising to and from the guest memory, and all guest (WASI) types are now proper idiomatic Rust types. While we're here, in preparation for the ephemeral snapshot, I went ahead and reorganised the internal structure of the crate. Instead of modules like `hostcalls_impl` or `hostcalls_impl::fs`, the structure now resembles that in ephemeral with modules like `path`, `fd`, etc. Now, I'm not requiring we leave it like this, but I reckon it looks cleaner this way after all. * Fix wig to use new first-class access to caller's mem * Ignore warning in proc_exit for the moment * Group unsafes together in args and environ calls * Simplify pwrite; more unsafe blocks * Simplify fd_read * Bundle up unsafes in fd_readdir * Simplify fd_write * Add comment to path_readlink re zero-len buffers * Simplify unsafes in random_get * Hide GuestPtr<str> to &str in path::get * Rewrite pread and pwrite using SeekFrom and read/write_vectored I've left the implementation of VirtualFs pretty much untouched as I don't feel that comfortable in changing the API too much. Having said that, I reckon `pread` and `pwrite` could be refactored out, and `preadv` and `pwritev` could be entirely rewritten using `seek` and `read_vectored` and `write_vectored`. * Add comment about VirtFs unsafety * Fix all mentions of FdEntry to Entry * Fix warnings on Win * Add aux struct EntryTable responsible for Fds and Entries This commit adds aux struct `EntryTable` which is private to `WasiCtx` and is basically responsible for `Fd` alloc/dealloc as well as storing matching `Entry`s. This struct is entirely private to `WasiCtx` and as such as should remain transparent to `WasiCtx` users. * Remove redundant check for empty buffer in path_readlink * Preserve and rewind file cursor in pread/pwrite * Use GuestPtr<[u8]>::copy_from_slice wherever copying bytes directly * Use GuestPtr<[u8]>::copy_from_slice in fd_readdir * Clean up unsafes around WasiCtx accessors * Fix bugs in args_get and environ_get * Fix conflicts after rebase |