* Update the top-level README.md and embedding documentation.
wasmtime-api is now the primary external API crate, so recommend that
instead of wasmtime-jit.
Also, enable wasmtime-api's C API by default, so that it shows up on
docs.rs, and to make it easier to use.
And, add basic embedding documentation and link to it from the
README.md. Credit to @yurydelendik for the content.
* Use the new wasm-c-api URL.
* Don't pass --features wasm-c-api, as it is now on by default.
This commit simplifies the build script slightly for generating tests by
doing a few dull refactorings:
* Leaves formatting to `rustfmt`
* Extract bulk of code execution into a top-level shared `run_wast`
function so each test is a one-liner
* Use `anyhow` for errors both in the script and in tests
Some wasi-common tests assume that stdin is never ready to be read, but
on CI stdin is closed so it's always ready to be read. Work around this
by guaranteeing that wasi-common tests always have an unreadable stdin
pipe by creating our own pipe.
This commit switches the beta/nightly tests to happen on Ubuntu instead
of macOS. Turns out GitHub Actions has scheduling limitations on macOS
that limit repositories to 5 concurrent jobs per repository, so let's
reduce the load a bit by running more builds on Linux than mac.
We needed a workaround when using wabt on old Linux distros due to using
old versions of `strtof`. With the switch to `wat`, we no longer need
these workarounds.
This commit merges [CraneStation/wasi-common] repo as a subdir of
this repo while preserving **all** of git history. There is an
initiative to pull `wasi-common` into [CraneStation/wasmtime], and
[CraneStation/wasmtime] becoming a monorepo. This came about for
several reasons with a common theme of convenience, namely,
having a monorepo:
1. cleans up the problem of dependencies (as we have seen first
hand with dependabot enabled, it can cause some grief)
2. completely removes the problem of syncing the closely dependent
repos (e.g., updating `wasi-common` with say a bugfix generally
implies creating a "sync" commit for pulling in the changes into
the "parent" repo, in this case, `wasmtime`)
3. mainly for the two reasons above, makes publishing to crates.io
easier
4. hopefully streamlines the process of getting the community
involved in contributing to `wasi-common` as now everything
is one place
[CraneStation/wasi-common]: https://github.com/CraneStation/wasi-common
[CraneStation/wasmtime]: https://github.com/CraneStation/wasmtime
* Initial checkin.
* Update to rust-lang libc.
* Add a .gitignore file.
* Factor out functions for cleaning up files and directories.
* Fix a typo in a comment.
* Print a "Success!" message if all tests passed.
* Factor out code for creating directories.
* Add wrappers around WASI functions.
These wrappers handle converting from &str to pointer+length and handle
unsafe.
* More refactoring.
* Refactor a fd_close helper.
* Move utility functions into a separate file.
* cargo update
* Add a basic test for random_get.
* Test that directories aren't resizable.
* Test clearing __WASI_RIGHT_PATH_FILESTAT_SET_SIZE.
Ensure that clearing __WASI_RIGHT_PATH_FILESTAT_SET_SIZE succeeds before
testing file truncation.
* cargo update
* Modularise tests for easier use with wasi-common crate
* Add a Code of Conduct and CONTRIBUTING.md.
* Fix typo
* Add testcase for fd_allocate
* Add positive test for fd_renumber
* Assert bufused in readlink_no_buffer testcase
* Add positive readlink testcase
* Add testcase for fd_seek and fd_tell
* Add fd_p{read, write} test
* Add README
* Add cases with trailing slashes to interesting_paths
* Split nofollow_errors testcase into two
* nofollow_errors now operators on symlinks to existing resources
* dangling_symlink covers danling symlinks tests
* Factor out a `create_file` helper function.
* Switch from the error crate to `std::io::Error::last_os_error()`.
* Use `create_file` in the readlink test too.
* Add a test for fd_filestat_set_*
* Minor refactoring
Add missing cleanup_file calls to file_pread_pwrite and
file_seek_tell.
* Add testcase for unbuffered fd_write; fixes#11
* Add testcase for path_rename
* Use the wasi crate.
Switch from depending on libc to depending on the new wasi crate to provide
the low-level WASI interfaces.
See also https://github.com/rust-lang/libc/pull/1461.
* Add a test for path_filestat_*
* Add a test for fd_readdir
* Use expect instead of unwrap
* Add a check for ino.
* Fix the build
* Don't assume a specific order of dirents
* Better test
* Test cookie value
* Fix file types
* Fix the test
* Fix the test
* Fix the test
* Cleanup
* Minor formatting tidying in README.md.
* Fix miscellaneous clippy warnings.
* Rename the crate to wasi-misc-tests.
* Update to wasi 0.7.0.
This switches from using the libc wasi bindings to using the wasi
crate's bindings. This eliminates a git dependency on libc, updates
to the new-style bindings which use Result where possible, and treats
functions that operate on raw file descriptors as unsafe.
* Add various tests for trailing-slash behavior.
* Sync new testcases with latest upstream
* Fix path_filestat testcase
* Add smoke test for fd_advise
This test is a true smoke test as it only tests whether issuing
an advise call to the host's kernel doesn't yield an error. The
consequence of issuing such a syscall is not tested.
* Check if CLOCK_MONOTONIC is actually monotonic
* Refactor the inequality assertions for more debuggable errors.
* Bump libc from 0.2.62 to 0.2.65
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.62 to 0.2.65.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.62...0.2.65)
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
* Fix compilation error
* Enable Actions and add rust.yml (#35)
* Enable Actions and add rust.yml
This commit enables Github Actions and adds corresponding configuration in rust.yml file.
* Update rust.yml
* Fix formatting
* Add empty .rustfmt.toml config file
* Add badge to README
* Update README
* Clean up Github Actions and README
* Add test case for `poll_oneoff` syscall (#38)
* Add test case for `poll_oneoff` syscall
This commit adds a test case for `poll_oneoff` syscall. In particular,
it builds on the excellent test use case provided by @dunnock in their
repo [poll_oneoff_tests] (thanks!), and tests:
* simple timeout
* stdin read with timeout
* fd read and fd write polls
[poll_oneoff_tests]: https://github.com/dunnock/poll_oneoff_tests
* Apply suggestions and negative test for bad fd
Co-authored-by: Maxim Vorobjov <maxim.vorobjov@gmail.com>
* Add smoke test for STDOUT/ERR readwrite poll
* Add comment on stdin/out/err
* Add a test for `*at`-style functions returning `ENOTDIR` when `dirfd` is not a dir.
* Remove misc_testsuite submodule
* Add "publish=false" to Cargo.toml; remove LICENSE
* fix Linux `isatty` implementation
* defer `WasiCtxBuilder` errors to `build()`; don't change API yet
This changes the fields on the builder to types that let the various `.arg()`, `.env()`, etc methods
infallible, so we don't have to worry about handling any errors till we actually build. This reduces
line noise when using a builder in a downstream application.
Deferring the processing of the builder fields also has the advantage of eliminating the opening and
closing of `/dev/null` for the default stdio file descriptors unless they're actually used by the
resulting `WasiCtx`.
Unicode errors when inheriting arguments and environment variables no longer cause a panic, but
instead go through `OsString`. We return `ENOTCAPABLE` at the end if there are NULs, or if UTF-8
conversion fails on Windows.
This also changes the bounds on some of the methods from `AsRef<str>` to `AsRef<[u8]>`. This
shouldn't break any existing code, but allows more flexibility when providing arguments. Depending
on the outcome of https://github.com/WebAssembly/WASI/issues/8 we may eventually want to require
these bytes be UTF-8, so we might want to revisit this later.
Finally, this fixes a tiny bug that could arise if we had exactly the maximum number of file
descriptors when populating the preopens.
* make `WasiCtxBuilder` method types less restrictive
This is a separate commit, since it changes the interface that downstream clients have to use, and
therefore requires a different commit of `wasmtime` for testing. That `wasmtime` commit is currently
on my private fork, so this will need to be amended before merging.
Now that failures are deferred until `WasiCtxBuilder::build()`, we don't need to have `Result` types
on the other methods any longer.
Additionally, using `IntoIterator` rather than `Iterator` as the trait bound for these methods is
slightly more general, and saves the client some typing.
* enforce that arguments and environment variables are valid UTF-8
* remove now-unnecessary platform-specific OsString handling
* `ENOTCAPABLE` -> `EILSEQ` for failed arg/env string conversions
* fix up comment style
* Apply @acfoltzer's fix to isatty on Linux to BSD
* Reimpleent fd_readdir on Linux using quasi-nix.
* Implement fd_readdir on Windows.
* wip
* Adapt to upstream changes.
* Cleanup dir.rs
* Address review
* Fix macos build
* host -> wasi
* Partially address review, more to come later.
* Address more review comments
* Fix compilation on Windows
This commit implements registering unwind information for JIT functions on
Windows so that the operating system can both walk and unwind stacks containing
JIT frames.
Currently this only works with Cranelift as lightbeam does not emit unwind
information yet.
This commit also resets the stack guard page on Windows for stack overflow
exceptions, allowing reliable stack overflow traps.
With these changes, all previously disabled test suite tests (not including
the multi-value tests) on Windows are now passing.
Fixes#291.
This commit bumps Cranelift to 0.49.0 and target-lexicon to 0.9.0 to fix the
failing build to due an updated faerie crate that violated semver with an
updated 0.9.0 target-lexicon dependency.
Fixes#491.
* Implement clock_time_get on Windows.
Also update misc_testsuite to include latest clock_time_get test
changes.
* Try to somehow implement clock_res_get on Windows.
* Fix 55ms
* Cache the perf counter resolution
* Fix integration tests
This commit migrates wasmtime's CI infrastructure from Azure Pipelines
to Github Actions. Using Github Actions has a few benefits over other
offerings:
* Being natively integrated with Github means that there's no degree of
user account configuration or access control management, it's all
inherent via already existing Github permissions.
* Github Actions gives 20 parallel builders instead of Azure's 10 by
default, which is a nice boost to have!
Overall I've found Github Actions to feel a bit cleaner than Azure
Pipelines as well. Subjectively I've found the configuration to be more
readable and more pleasant to work with, although they're both just as
"powerful" I think. Additionally Github Actions has been pretty solid in
my own personal testing for a number of other projects.
The main trickiness with wasmtime's CI is the rolling `dev` release of
the master branch as well as binary releases for tags. Github Actions
doesn't have quite as much built in functionality as Azure Pipelines,
but Github Actions does have a nice feature where you can define the
code for an action locally rather than only using built-in actions.
This migration adds three local actions with some associated JS code to
run the action (currently it looks like it basically requires JS)
* An `install-rust` action papers over the gotchas about installing
Rust, allowing Rust installation to be a one-liner in the configuration.
* A `binary-compatible-builds` action allows easily configuring the
wheels and the binaries to be "more binary compatible" and handles
things like compilation flags on OSX and Windows while handling the
`centos:6` container on Linux.
* The `github-release` action is the logic using the `@actions/github`
JS package to orchestrate the custom way we manage rolling releases,
ensuring that a new release is made for the master branch under `dev`
(deleting the previous tag/release ahead of time) and then also
manages tagged releases by uploading them there.
I'm hoping that most of the inline actions here will largely go away.
For example `install-rust` should be simply `rustup update $toolchain`
once various environment issues are fixed on Github Actions runner
images. Additionally `github-release` will ideally migrate to something
like https://github.com/actions/create-release or similar once it has
enough functionality. I'm also hoping that the maintenance in the
meantime of these actions is pretty low-cost, but if it becomes an issue
we can look into other solutions!
* Bump crate versions.
* Update dependency version numbers too.
This is a follow-up to f96b6c9e72ae50fcddd488be47a2d4b5ac7b926b to
update the version numbers in the local dependencies too.
* Use generated type bindings.
Use the witx API descriptions to generate the bulk of the contents of
host.rs, wasi.rs, and wasi32.rs.
This also prunes out many of the miscellaneous libc definitions from
those files which aren't currently in use by wasi-common. If there's
anything removed that's still needed by someone, it's easy to add things
back in.
* Remove unneeded iovec conversion routines.
* Migrate from failure to thiserror and anyhow
The failure crate invents its own traits that don't use
std::error::Error (because failure predates certain features added to
Error); this prevents using ? on an error from failure in a function
using Error. The thiserror and anyhow crates integrate with the standard
Error trait instead.
This change does not attempt to semantically change or refactor the
approach to error-handling in any portion of the code, to ensure that
the change remains straightforward to review. Modules using specific
differentiated error types move from failure_derive and derive(Fail) to
thiserror and derive(Error). Modules boxing all errors opaquely move
from failure::Error to anyhow. Modules using String as an error type
continue to do so. Code using unwrap or expect continues to do so.
Drop Display implementations when thiserror can easily derive an
identical instance.
Drop manual traversal of iter_causes; anyhow's Debug instance prints the
chain of causes by default.
Use anyhow's type alias anyhow::Result<T> in place of
std::result::Result<T, anyhow::Error> whenever possible.
* wasm2obj: Simplify error handling using existing messages
handle_module in wasm2obj manually maps
cranelift_codegen::isa::LookupError values to strings, but LookupError
values already have strings that say almost exactly the same thing.
Rely on the strings from cranelift.
* wasmtime: Rely on question-mark-in-main
The main() wrapper around rmain() completely matches the behavior of
question-mark-in-main (print error to stderr and return 1), so switch to
question-mark-in-main.
* Update to walrus 0.13 and wasm-webidl-bindings 0.6
Both crates switched from failure to anyhow; updating lets us avoid a
translation from failure to anyhow within wasmtime-interface-types.
This commit adds a utility routine
`strip_trailing_slashes_and_concatenate` which is common for
`path_rename` and `path_symlink` on Windows, and checks if the resolved
`PathGet` indeed contains a trailing slash(es) before striping them
off. Secondly, this commit fixes `path_rename_trailing_slashes` test
case by adding two additional checks for potentially erroneous
conditions, and raising `ENOTDIR` if any happens to be true.
Since `wasmtime` now uses `wasi` and `wasi32` modules, we can now
safely remove the `wasm32` module. This commit also updates `wasmtime`
to the latest upstream.
This commit syncs `wasmtime-wasi` crate with the latest refactoring
applied to `wasi-common` crate. Namely, `wasm32` is replaced with
two modules: `wasi` and `wasi32`. This change can be tracked via
CraneStation/wasi-common#151.
This commit fixes incorrect rights check in `fd_pwrite`. Until now,
we were erroneously checking whether the `Descriptor` has
`__WASI_RIGHT_FD_READ` rights instead of `__WASI_RIGHT_FD_WRITE`.
Additionally, this commit removes redundant borrows from
`wasi_ctx.get_fd_entry(..)` calls.
* Reorganize host.rs and wasm32.rs.
Reorganize host.rs and wasm32.rs into host.rs, wasi.rs, and wasi32.rs.
Most of the contents of host.rs was not actually host-specific, as most
of the types are fixed-size types like u32 or i64. These types are now
in wasi.rs.
The few types which do have pointer or usize-sized values now remain,
in two versions: host.rs has versions which use actual raw pointers and
usize, and wasi32.rs has versions which use u32 to represent them.
* Fix compilation on BSD
* Fix compilation on Windows
* Fully encapsulate endianness in memory.rs.
This refactors memory.rs to fully encapsulte endianness concerns, so
that outside that file, all values are in host-endian order.
This adds a dependency on the `num` crate, though it's only used for
the `PrimInt` trait, for handling endianness in a generic way.
* Use pub(crate).
Unfortunately the helpers added in #154 were only used from non-Windows
implementations, which caused compiler warnings on Windows.
This commit moves the helper to be unix-specific and removes the tiny wrapper
around calling `str_to_cstring` on `PathGet.path`.
* Fix path_get returning ENOTDIR when base not dir
This commit makes certain adjustments to `WasiCtx` and `FdEntry`
by introducing methods `get_nonvalidated_fd_entry` and
`get_nonvalidated_fd_entry_mut` which only check if the `FdEntry`
corresponding to the specified raw WASI fd exists in the context
object but **without** automatically validating the rights. Thanks
to postponing the validation until after the `FdEntry` object is
extracted from the context, it is possible to check if the extracted
`FdEntry` object is indeed a valid dir descriptor when processing
paths in `path_get` helper fn. In essence then, this commit closes
#158.
* Remove potentially useless FdObject struct
This commit removes `FdObject` struct which wasn't really used
in our codebase, and flattens its contents into `FdEntry`. IMHO,
this cleans up a fair amount of code and generally unclutters it.
* Refactor and document `WasiCtx` and `FdEntry`
This commit refactors `WasiCtx` struct by making the collection
of `FdEntry`s stored within private, and only allowing it to be
accessed via the provided set of modifier methods (push, insert,
remove, etc.).
Additionally, this commit documents the methods of `WasiCtx` and
`FdEntry` structs for easier debugging and maintenance in the future.
Return `ENOTDIR` instead of `ENOTCAPABLE` when `*at` functions are
given a non-directory as their base. This is in accordance with POSIX:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html
Currently wasi-libc contains some code to detect such cases and rewrite
`ENOTCAPABLE` to `ENOTDIR`, however it's better for WASI implementations
to just do the right thing in the first place.
* FixesCraneStation/wasmtime#440
This commit introduces a couple of changes/fixes:
* it annotates `log::debug!` messages with "host" to differentiate
between file descriptors stored on the WASI side (aka the wrappers)
and those managed by the host (aka the wrapped)
* it fixesCraneStation/wasmtime#440, i.e., incorrect passing of
file descriptor to `poll_oneoff` where currently errenously we
pass in the wrapper instead of the wrapped value
* it adds a couple more `log::debug!` macros calls for easier future
debugging
* Add partial refactorting to poll_oneoff
This commit lays the groundwork for more clean up to come in
subsequent commits.
* Finalise refactoring of `poll_oneoff`
* Fix compilation error on Windows
* Address majority of suggestions and refactor
Co-authored-by: Marcin Mielniczuk <marmistrz.dev@zoho.eu>
* Add poll_oneoff test case
* Leave timeout in nanoseconds in ClockEventData
Instead of converting the timeout value from nanoseconds to
milliseconds in the host-independent impl, move the conversion
to *nix-specific impl as the conversion is currently only warranted
by the POSIX `poll` syscall.
* Don't fail immediately on bad descriptor
If the user specifies an invalid descriptor inside a subscription,
don't fail immediately but rather generate an event with the thrown
WASI error code, and continue with the remaining, potentially
correct subscriptions.
This commit implements a simple helper for converting `&str` to `CString` and
mapping to the appropriate WASI error.
It also adds a `path_cstring` helper method in `PathGet` where the conversion was
used the most.
Fixes#104.
Instead of returning the debug formatting, which includes the enum
names, like `Io(...)`, just run the formatting function of the inner
error objects, which is nicer for command-line use.
* [wasmtime-api] run example as tests
* Update wasmtime-api/tests/examples.rs
Co-Authored-By: Nick Fitzgerald <fitzgen@gmail.com>
* disable multi example test on windows
This commit adds the skeleton of a new set of documentation for
`wasmtime` in the existing `docs` directory. This documentation is
organized and compiled with [mdbook] which the Rust project uses for
most of its own documentation as well. At a previous meeting we
brainstormed a rough skeleton of what the documentation in this book
would look like, and I've transcribed that here for an example of how
this is rendered and how it can be laid out. No actual documentation is
written yet.
This commit also additionally adds necessary support to auto-publish
both this book documentation and API documentation every time a commit
is pushed to the `master` branch. All HTML will be automatically pushed
to the `gh-pages` branch so long as the CI passes, and this should get
deployed to https://cranestation.github.io/wasmtime.
I've done a few dry-runs and I think this'll all work, but we'll likely
tweak a few things here and there after running this through CI to make
sure everything looks just as we'd like. My hope though is that after
this lands we can start actually filling out all the documentation and
being able to review it as well.
[mdbook]: https://crates.io/crates/mdbook
cargo test will test examples that have a "main.rs", but in this case,
the example requires manual compilation as a wasm module, and doesn't
build as a standalone program. Rename it to "demo.rs".
