Commit Graph

232 Commits

Author SHA1 Message Date
Jakub Konka
3793fa3b09 Merge wasi-misc-tests repo as a subdir (#174)
* Initial checkin.

* Update to rust-lang libc.

* Add a .gitignore file.

* Factor out functions for cleaning up files and directories.

* Fix a typo in a comment.

* Print a "Success!" message if all tests passed.

* Factor out code for creating directories.

* Add wrappers around WASI functions.

These wrappers handle converting from &str to pointer+length and handle
unsafe.

* More refactoring.

* Refactor a fd_close helper.

* Move utility functions into a separate file.

* cargo update

* Add a basic test for random_get.

* Test that directories aren't resizable.

* Test clearing __WASI_RIGHT_PATH_FILESTAT_SET_SIZE.

Ensure that clearing __WASI_RIGHT_PATH_FILESTAT_SET_SIZE succeeds before
testing file truncation.

* cargo update

* Modularise tests for easier use with wasi-common crate

* Add a Code of Conduct and CONTRIBUTING.md.

* Fix typo

* Add testcase for fd_allocate

* Add positive test for fd_renumber

* Assert bufused in readlink_no_buffer testcase

* Add positive readlink testcase

* Add testcase for fd_seek and fd_tell

* Add fd_p{read, write} test

* Add README

* Add cases with trailing slashes to interesting_paths

* Split nofollow_errors testcase into two

* nofollow_errors now operators on symlinks to existing resources
* dangling_symlink covers danling symlinks tests

* Factor out a `create_file` helper function.

* Switch from the error crate to `std::io::Error::last_os_error()`.

* Use `create_file` in the readlink test too.

* Add a test for fd_filestat_set_*

* Minor refactoring

Add missing cleanup_file calls to file_pread_pwrite and
file_seek_tell.

* Add testcase for unbuffered fd_write; fixes #11

* Add testcase for path_rename

* Use the wasi crate.

Switch from depending on libc to depending on the new wasi crate to provide
the low-level WASI interfaces.

See also https://github.com/rust-lang/libc/pull/1461.

* Add a test for path_filestat_*

* Add a test for fd_readdir

* Use expect instead of unwrap

* Add a check for ino.

* Fix the build

* Don't assume a specific order of dirents

* Better test

* Test cookie value

* Fix file types

* Fix the test

* Fix the test

* Fix the test

* Cleanup

* Minor formatting tidying in README.md.

* Fix miscellaneous clippy warnings.

* Rename the crate to wasi-misc-tests.

* Update to wasi 0.7.0.

This switches from using the libc wasi bindings to using the wasi
crate's bindings. This eliminates a git dependency on libc, updates
to the new-style bindings which use Result where possible, and treats
functions that operate on raw file descriptors as unsafe.

* Add various tests for trailing-slash behavior.

* Sync new testcases with latest upstream

* Fix path_filestat testcase

* Add smoke test for fd_advise

This test is a true smoke test as it only tests whether issuing
an advise call to the host's kernel doesn't yield an error. The
consequence of issuing such a syscall is not tested.

* Check if CLOCK_MONOTONIC is actually monotonic

* Refactor the inequality assertions for more debuggable errors.

* Bump libc from 0.2.62 to 0.2.65

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.62 to 0.2.65.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.62...0.2.65)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Fix compilation error

* Enable Actions and add rust.yml (#35)

* Enable Actions and add rust.yml

This commit enables Github Actions and adds corresponding configuration in rust.yml file.

* Update rust.yml

* Fix formatting

* Add empty .rustfmt.toml config file

* Add badge to README

* Update README

* Clean up Github Actions and README

* Add test case for `poll_oneoff` syscall (#38)

* Add test case for `poll_oneoff` syscall

This commit adds a test case for `poll_oneoff` syscall. In particular,
it builds on the excellent test use case provided by @dunnock in their
repo [poll_oneoff_tests] (thanks!), and tests:
* simple timeout
* stdin read with timeout
* fd read and fd write polls

[poll_oneoff_tests]: https://github.com/dunnock/poll_oneoff_tests

* Apply suggestions and negative test for bad fd

Co-authored-by: Maxim Vorobjov <maxim.vorobjov@gmail.com>

* Add smoke test for STDOUT/ERR readwrite poll

* Add comment on stdin/out/err

* Add a test for `*at`-style functions returning `ENOTDIR` when `dirfd` is not a dir.

* Remove misc_testsuite submodule

* Add "publish=false" to Cargo.toml; remove LICENSE
2019-11-07 13:58:57 +01:00
Adam C. Foltzer
2fe353044f Improvements to WasiCtxBuilder, and a couple bug fixes (#175)
* fix Linux `isatty` implementation

* defer `WasiCtxBuilder` errors to `build()`; don't change API yet

This changes the fields on the builder to types that let the various `.arg()`, `.env()`, etc methods
infallible, so we don't have to worry about handling any errors till we actually build. This reduces
line noise when using a builder in a downstream application.

Deferring the processing of the builder fields also has the advantage of eliminating the opening and
closing of `/dev/null` for the default stdio file descriptors unless they're actually used by the
resulting `WasiCtx`.

Unicode errors when inheriting arguments and environment variables no longer cause a panic, but
instead go through `OsString`. We return `ENOTCAPABLE` at the end if there are NULs, or if UTF-8
conversion fails on Windows.

This also changes the bounds on some of the methods from `AsRef<str>` to `AsRef<[u8]>`. This
shouldn't break any existing code, but allows more flexibility when providing arguments. Depending
on the outcome of https://github.com/WebAssembly/WASI/issues/8 we may eventually want to require
these bytes be UTF-8, so we might want to revisit this later.

Finally, this fixes a tiny bug that could arise if we had exactly the maximum number of file
descriptors when populating the preopens.

* make `WasiCtxBuilder` method types less restrictive

This is a separate commit, since it changes the interface that downstream clients have to use, and
therefore requires a different commit of `wasmtime` for testing. That `wasmtime` commit is currently
on my private fork, so this will need to be amended before merging.

Now that failures are deferred until `WasiCtxBuilder::build()`, we don't need to have `Result` types
on the other methods any longer.

Additionally, using `IntoIterator` rather than `Iterator` as the trait bound for these methods is
slightly more general, and saves the client some typing.

* enforce that arguments and environment variables are valid UTF-8

* remove now-unnecessary platform-specific OsString handling

* `ENOTCAPABLE` -> `EILSEQ` for failed arg/env string conversions

* fix up comment style

* Apply @acfoltzer's fix to isatty on Linux to BSD
2019-11-07 11:50:47 +01:00
Marcin Mielniczuk
12972c7fd3 Implement fd_readdir (#116)
* Reimpleent fd_readdir on Linux using quasi-nix.

* Implement fd_readdir on Windows.

* wip

* Adapt to upstream changes.

* Cleanup dir.rs

* Address review

* Fix macos build

* host -> wasi

* Partially address review, more to come later.

* Address more review comments

* Fix compilation on Windows
2019-11-07 11:23:52 +01:00
Marcin Mielniczuk
22494057df Try to somehow implement clock_res_get on Windows. (#124)
* Implement clock_time_get on Windows.

Also update misc_testsuite to include latest clock_time_get test
changes.

* Try to somehow implement clock_res_get on Windows.

* Fix 55ms

* Cache the perf counter resolution

* Fix integration tests
2019-11-06 17:02:58 +01:00
Dan Gohman
3757b8b1c2 Update local dependency versions too (#173)
* Bump crate versions.

* Update dependency version numbers too.

This is a follow-up to f96b6c9e72ae50fcddd488be47a2d4b5ac7b926b to
update the version numbers in the local dependencies too.
2019-11-05 15:31:39 -08:00
Dan Gohman
ab0c933eff Bump crate versions. (#171) 2019-11-05 15:13:13 -08:00
Dan Gohman
8ebe12f553 Use generated type bindings (#152)
* Use generated type bindings.

Use the witx API descriptions to generate the bulk of the contents of
host.rs, wasi.rs, and wasi32.rs.

This also prunes out many of the miscellaneous libc definitions from
those files which aren't currently in use by wasi-common. If there's
anything removed that's still needed by someone, it's easy to add things
back in.

* Remove unneeded iovec conversion routines.
2019-11-05 14:56:18 -08:00
Jakub Konka
f4ac1299b2 Fix path_rename_trailing_slashes test case on Win
This commit adds a utility routine
`strip_trailing_slashes_and_concatenate` which is common for
`path_rename` and `path_symlink` on Windows, and checks if the resolved
`PathGet` indeed contains a trailing slash(es) before striping them
off. Secondly, this commit fixes `path_rename_trailing_slashes` test
case by adding two additional checks for potentially erroneous
conditions, and raising `ENOTDIR` if any happens to be true.
2019-11-04 22:35:51 +01:00
Jakub Konka
74757fdd65 Sync wasmtime and remove wasm32 deprecated module (#165)
Since `wasmtime` now uses `wasi` and `wasi32` modules, we can now
safely remove the `wasm32` module. This commit also updates `wasmtime`
to the latest upstream.
2019-11-02 06:33:54 -07:00
Dan Gohman
95578a42ce Remove RIGHTS_SHARED_MEMORY_BASE.
This came from CloudABI, but WASI doesn't include POSIX-style
shared-memory objects.
2019-11-02 09:10:33 +01:00
Dan Gohman
86829d2447 Handle overflow in Linux's dirent_from_host. 2019-11-02 08:53:30 +01:00
Jakub Konka
37ce4ba2ad Fix rights in fd_pwrite; cleanup redundant borrows
This commit fixes incorrect rights check in `fd_pwrite`. Until now,
we were erroneously checking whether the `Descriptor` has
`__WASI_RIGHT_FD_READ` rights instead of `__WASI_RIGHT_FD_WRITE`.

Additionally, this commit removes redundant borrows from
`wasi_ctx.get_fd_entry(..)` calls.
2019-11-01 22:56:32 +01:00
Dan Gohman
a9e3487566 Reorganize host.rs and wasm32.rs. (#151)
* Reorganize host.rs and wasm32.rs.

Reorganize host.rs and wasm32.rs into host.rs, wasi.rs, and wasi32.rs.

Most of the contents of host.rs was not actually host-specific, as most
of the types are fixed-size types like u32 or i64. These types are now
in wasi.rs.

The few types which do have pointer or usize-sized values now remain,
in two versions: host.rs has versions which use actual raw pointers and
usize, and wasi32.rs has versions which use u32 to represent them.

* Fix compilation on BSD

* Fix compilation on Windows

* Fully encapsulate endianness in memory.rs.

This refactors memory.rs to fully encapsulte endianness concerns, so
that outside that file, all values are in host-endian order.

This adds a dependency on the `num` crate, though it's only used for
the `PrimInt` trait, for handling endianness in a generic way.

* Use pub(crate).
2019-11-01 14:21:32 -07:00
Jakub Konka
5f5f31beab Treat all warnings as errors by default (#161)
* Treat all warnings as errors in the CI

* Build only in release mode
2019-10-31 11:45:42 +01:00
Peter Huene
3b7a00f474 Fix unused function warnings on Windows.
Unfortunately the helpers added in #154 were only used from non-Windows
implementations, which caused compiler warnings on Windows.

This commit moves the helper to be unix-specific and removes the tiny wrapper
around calling `str_to_cstring` on `PathGet.path`.
2019-10-30 23:04:27 +01:00
Jakub Konka
f4d3f08fc6 Fix path_get returning ENOTDIR when base not dir (#159)
* Fix path_get returning ENOTDIR when base not dir

This commit makes certain adjustments to `WasiCtx` and `FdEntry`
by introducing methods `get_nonvalidated_fd_entry` and
`get_nonvalidated_fd_entry_mut` which only check if the `FdEntry`
corresponding to the specified raw WASI fd exists in the context
object but **without** automatically validating the rights. Thanks
to postponing the validation until after the `FdEntry` object is
extracted from the context, it is possible to check if the extracted
`FdEntry` object is indeed a valid dir descriptor when processing
paths in `path_get` helper fn. In essence then, this commit closes
 #158.

* Remove potentially useless FdObject struct

This commit removes `FdObject` struct which wasn't really used
in our codebase, and flattens its contents into `FdEntry`. IMHO,
this cleans up a fair amount of code and generally unclutters it.

* Refactor and document `WasiCtx` and `FdEntry`

This commit refactors `WasiCtx` struct by making the collection
of `FdEntry`s stored within private, and only allowing it to be
accessed via the provided set of modifier methods (push, insert,
remove, etc.).

Additionally, this commit documents the methods of `WasiCtx` and
`FdEntry` structs for easier debugging and maintenance in the future.
2019-10-30 14:41:43 -07:00
Dan Gohman
0302f1a164 Make path_get return ENOTDIR when the base isn't a directory.
Return `ENOTDIR` instead of `ENOTCAPABLE` when `*at` functions are
given a non-directory as their base. This is in accordance with POSIX:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html

Currently wasi-libc contains some code to detect such cases and rewrite
`ENOTCAPABLE` to `ENOTDIR`, however it's better for WASI implementations
to just do the right thing in the first place.
2019-10-30 10:20:42 +01:00
Jakub Konka
f3a5186230 Refactor poll_oneoff on *nix (#137)
* Fixes CraneStation/wasmtime#440

This commit introduces a couple of changes/fixes:
* it annotates `log::debug!` messages with "host" to differentiate
  between file descriptors stored on the WASI side (aka the wrappers)
  and those managed by the host (aka the wrapped)
* it fixes CraneStation/wasmtime#440, i.e., incorrect passing of
  file descriptor to `poll_oneoff` where currently errenously we
  pass in the wrapper instead of the wrapped value
* it adds a couple more `log::debug!` macros calls for easier future
  debugging

* Add partial refactorting to poll_oneoff

This commit lays the groundwork for more clean up to come in
subsequent commits.

* Finalise refactoring of `poll_oneoff`

* Fix compilation error on Windows

* Address majority of suggestions and refactor

Co-authored-by: Marcin Mielniczuk <marmistrz.dev@zoho.eu>

* Add poll_oneoff test case

* Leave timeout in nanoseconds in ClockEventData

Instead of converting the timeout value from nanoseconds to
milliseconds in the host-independent impl, move the conversion
to *nix-specific impl as the conversion is currently only warranted
by the POSIX `poll` syscall.

* Don't fail immediately on bad descriptor

If the user specifies an invalid descriptor inside a subscription,
don't fail immediately but rather generate an event with the thrown
WASI error code, and continue with the remaining, potentially
correct subscriptions.
2019-10-30 09:57:59 +01:00
Dan Gohman
aecec187d7 Check for overflow when incrementing the preopen_fd counter. (#157)
If someone somehow defines more preopens than can fit in the
`__wasi_fd_t` index space, we should detect it instead of silently
wrapping around.
2019-10-29 17:11:43 -07:00
Peter Huene
de71dbec0d Implement a helper for converting &str to CString. (#154)
This commit implements a simple helper for converting `&str` to `CString` and
mapping to the appropriate WASI error.

It also adds a `path_cstring` helper method in `PathGet` where the conversion was
used the most.

Fixes #104.
2019-10-29 16:30:22 -07:00
Dan Gohman
9a5c53b3ed Make wasi-common's Error messages prettier. (#156)
Instead of returning the debug formatting, which includes the enum
names, like `Io(...)`, just run the formatting function of the inner
error objects, which is nicer for command-line use.
2019-10-29 15:46:11 -07:00
Marcin Mielniczuk
2659641132 Implement clock_time_get on Windows (#119)
* Fix some Windows warnings.

* Implement clock_time_get on Windows.

Also update misc_testsuite to include latest clock_time_get test
changes.

* improve comments

* Remove a leftover import.

Co-Authored-By: Jakub Konka <kubkon@jakubkonka.com>
2019-10-29 15:05:42 +01:00
Jakub Konka
bb5c879718 Fixes path_symlink_trailing_slashes on Windows
This commit:
* adds missing `ERROR_ALREADY_EXISTS => __WASI_EEXIST` mapping
* re-routes Win errors into correct WASI values in `symlink_*`
  fns when target exists and/or contains a trailing slash
* remaps `ERROR_INVALID_NAME => __WASI_ENOENT`
2019-10-29 11:04:16 +01:00
Jakub Konka
59bbfbc0d7 Remove needs_close from FdObject
This commit removes `needs_close` field from `FdObject`, and
stores the underlying `Descriptor` without the `ManuallyDrop`
wrapper.
2019-10-25 17:39:47 +02:00
Maxim Vorobjov
6eb6e342d7 Rebuild misc tests when any of test files change (#150)
* rebuild misc tests when any of test file changes

* fix build formatting

* file change validation -> build_and_generate_tests
2019-10-25 15:48:58 +02:00
Pat Hickey
f9a8329424 WasiCtxBuilder: add methods to provide Files each for stdio (#147) 2019-10-25 14:23:26 +02:00
Jakub Konka
b0e896e7d0 Add Github Actions badge to README 2019-10-25 11:52:18 +02:00
Jakub Konka
1c5422e052 Remove Azure Pipelines conf 2019-10-25 11:52:18 +02:00
Jakub Konka
997d411870 Bump required rustc to 1.37 in README 2019-10-25 11:52:18 +02:00
Jakub Konka
3821ce28c8 Add beta & nightly builds 2019-10-25 11:52:18 +02:00
Jakub Konka
2ada299b95 Add docs section 2019-10-25 11:52:18 +02:00
Jakub Konka
a04aa5860d Update test configuration 2019-10-25 11:52:18 +02:00
Jakub Konka
10e0b87ebf Update MSRV to 1.37.0 2019-10-25 11:52:18 +02:00
Jakub Konka
be1f8c0325 Update ci.yml 2019-10-25 11:52:18 +02:00
Jakub Konka
b0199ef396 Enable Github Actions 2019-10-25 11:52:18 +02:00
Jakub Konka
8e45a14cec Fast-forward wasmtime to latest rev 2019-10-24 12:07:45 +02:00
Jakub Konka
3a374d0016 Update poll_oneoff API
This commit updates `poll_oneoff`'s API in a potentially least
invasive way. That is, it adds unused `WasiCtx` argument to the
syscall which will be required by #137. I am hopeful that this way
 #137 can pass all tests and hence this commit should aid the review
 process.
2019-10-24 11:19:33 +02:00
Dan Gohman
bd8d550ba3 Improve Cargo.toml files for publishing (#141)
* Add versions to the crates.

This is needed for publishing on crates.io.

* Add a few Cargo.toml attributes to make the crates.io page friendlier.
2019-10-23 19:17:22 +02:00
Dan Gohman
056e1d8cc0 Add more links in the fs module's documentation. 2019-10-23 19:16:18 +02:00
Jakub Konka
951456d79b Update misc_testsuite to latest master 2019-10-23 19:13:05 +02:00
Jakub Konka
63c1f71036 Sync with latest wasmtime rev
This commit syncs tests with latest wasmtime revision.
As such, it now utilises the `wasmtime-api` crate for
runtime setup.

Closes #126, #127, #128, #129.
2019-10-22 15:27:21 +02:00
Jakub Konka
0d63cc2dbc Fix remove_directory_trailing_slashes on Windows
This commit provides a fix for `remove_directory_trailing_slashes`
test case on Windows. It adds a missing mapping between the following
WinAPI error code and WASI error:

```
ERROR_DIRECTORY => __WASI_ENOTDIR
```

where `ERROR_DIRECTORY` is thrown when the directory name is invalid.
2019-10-22 10:12:54 +02:00
Dan Gohman
ef010b44b7 Fix nondeterminism in the preopen order.
We iterate over the preopens to present them to the WASI program, so
storing them in a `HashMap` means this order is nondeterministic. Switch
to a `Vec` of tuples instead. This means we don't eliminate duplicates,
but they should be rare.
2019-10-18 19:52:08 +02:00
Dan Gohman
4120d3b44f Optimize away an owned string in path_readlink. 2019-10-18 19:37:58 +02:00
Dan Gohman
14bad3cb97 Ensure that each crate has its own copy of the LICENSE file.
On crates.io, each crate is distributed separately, so ensure that each
crate has a license file.
2019-10-18 19:27:36 +02:00
Jakub Konka
c3bf04042e Fixes path_symlink_trailing_slashes test case (#125)
* Fixes `path_symlink_trailing_slashes` test case

This commit:
* adds a couple `log::debug!` macro calls in and around `path_get`
  for easier future debugging
* changes impl of `path_symlink` hostcall to actually *require*
  the final component (matching the impl of WASI in C)
* ignores the error `__WASI_ENOTDIR` in `path_get`'s `readlinkat` call
  which is not meant to be an error at this stage (i.e., this
  potentially erroneous condition *will be* handled later, in
  one of the layers above)

* Fixes `path_symlink_trailing` slashes on BSD-nixes

This commit:
* makes `path_symlink` host-specific (Linux and BSD-like nixes
  now have their own differing implementations)
* on BSD-like nixes, when `ENOTDIR` is returned from `symlinkat`
  it checks whether the target path contains a trailing slash,
  strips it, and then checks if the target path without the trailing
  slash exists; if yes, then converts the error code to `EEXIST` to
  match Linux/POSIX spec
2019-10-17 11:33:56 +02:00
Jakub Konka
d458fb6815 Clean up BSD vs Linux implementation details
This commit moves a couple of things around:
* separates the logic of `path_unlink_file` into separate impls
  for linux and BSD-style nixes
* moves implementation consts into appropriate impl modules: linux
  or bsd
* cleans up `utime_now` and `utime_omit` for BSD-style nixes
2019-10-17 11:15:01 +02:00
Jakub Konka
f1f6b3780b Fixes CraneStation/wasmtime#396
This commit fixes an issue with incorrect handling of /dev/(u)random
on Linux. It turns out that `nix::unistd::isatty` call handled only
the POSIX spec case where `ENOTTY` is returned in case the passed
in file descriptor is OK but not a TTY, whereas on Linux this is not
always the case. On Linux, it can be the case that `EINVAL` is returned
instead and this case AFAIK is not handled by the `nix` crate. This
commit fixes this by using `libc::isatty` syscall directly and checking
the return values.
2019-10-17 10:38:58 +02:00
Marcin Mielniczuk
968fb19f17 Add a typesafe enum for file type 2019-10-15 19:25:05 +09:00
Marcin Mielniczuk
5dad532a43 Use cvt from the cvt crate in winx 2019-10-15 17:15:10 +09:00