Currently our Linux binaries aren't quite as portable as they otherwise could
be. There's two primary reasons for this, and one of them is that the binary is
produced in a relatively recent Linux distribution (Ubuntu 16.04) which means
it has a relatively recent requirement in terms of glibc versions. On
OSX/Windows we can set some flags to rely on older libc implementations, but on
Linux we have to actually build against an older version.
This commit switches the container for the build to CentOS 6 instead of the
default Ubuntu 16.04. The main trick here is also finding a C++11-capable
compiler to compile wabt. Turns out though there's a helpful tutorial for this
at https://edwards.sdsu.edu/research/c11-on-centos-6/ and it was as easy as
installing a few packages.
The second portability concern of our Linux binaries is that they link
dynamically to `libstdc++.so` which isn't always installed on target systems,
or even if it is it may be too old or have a different ABI. This is solved by
statically linking to `libstdc++.a` in the build on Azure by doing a bit of
trickery with libraries and what's available.
After these results the glibc requirements drops from 2.18 (released in 2013)
to 2.6 (released in 2007) and avoids the need for users to have libstdc++.so
installed. We may eventually want to investigate fully-static musl binaries,
but finding a musl compiler for C++ is something I'm not that good at, so I
figure this is probably good enough for now.
* Remove cmake/bindgen/llvm from wasmtime-runtime
This commit removes the cmake/bindgen dependency (which removes the need
for `llvm-config`) from the `wasmtime-runtime` crate. The C++ code is
instead compiled with the `cc` crate (it's just one file anyway) and the
interface is handwritten since it's quite small anyway.
Some other changes are:
* The `TrapContext` type in C++ was removed since it was unused, and it
was moved to Rust with a `Cell` on each field.
* Functions between Rust/C++ now return `int` instead of `bool` to make
them a bit more FFI compatible portably.
* The `jmp_buf` type has a workaround that will be fixed in the next commit.
* Move setjmp/longjmp to C++
This commit moves the definition of setjmp and longjmp into C++. This is
primarily done because it's [debatable whether it's possible to call
`setjmp` from Rust][rfc]. The semantics of `setjmp` are that it returns
twice but LLVM doesn't actually know about this because rustc isn't
telling LLVM this information, so it's unclear whether it can ever be
safe.
Additionally this removes the need for Rust code to know the definition
of `jmp_buf` which is a pretty hairy type to define in Rust across
platforms.
The solution in this commit is to move all setjmp/longjmp code to C++,
and that way we should be able to guarantee that jumps over wasm JIT
code should always go from C++ to C++, removing Rust from the equation
for now from needing to get any fiddly bits working across platforms.
This should overall help it be a bit more portable and also means Rust
doesn't have to know about `jmp_buf` as a type.
The previous `Vec` of `jmp_buf` is now replaced with one thread-local
pointer where previous values are stored on the stack and restored when
the function returns. This is intended to be functionally the same as
the previous implementation.
[rfc]: https://github.com/rust-lang/rfcs/issues/2625
* rustfmt
* Use volatile loads/stores
* Remove mention of cmake from README
While the "__wasi_memory" name is something we considered, the name
currently being used for the memory exported to WASI is "memory", so
adjust the error message accordingly.
* Move path_get outside of sys module
* Add implementation of readlinkat
* Clean up path_open; use OpenOptions as much as possible
* Enable close_preopen test
* Implement path_create_directory; fix path_open
* Refactor path concatenation onto a descriptor
* Implement path_remove_directory
* Implement path_unlink_file
* Rewrite path_open using specific access mask
* Fix error mapping when unlinking file
* Fix readlinkat to pass nofollow_errors testcase
* Clean up winerror to WASI conversion
* Spoof creating dangling symlinks on windows (hacky!)
* Add positive testcase for readlink
* Implement path_readlink (for nonzero buffers for now)
* Clean up
* Add Symlink struct immitating *nix symlink
* Fix path_readlink
* Augment interesting_paths testcase with trailing slashes example
* Encapsulate path_get return value as PathGet struct
* Remove dangling symlink emulation
* Extract dangling symlinks into its own testcase
This way, we can re-enable nofollow_errors testcase
on Windows also.
* Return __WASI_ENOTCAPABLE if user lacks perms to symlink
* Simple module compilation cache
* Fix base64 encoding bug
* Use warn! everywhere in cache system
* Remove unused import
* Temporary workaround for long path on Windows
* Remove unused import for non-windows builds
* Add command line argument to enable cache system + apply minor review feedback
* Initial implementation of partial module hashing
* Proper module hashing for the cache
* Use newer version of cranelift
This commit compiles all Rust code for the Windows release with
`-Ctarget-feature=+crt-static`. This is targeted at increasing the
binary compatibility of the binaries built to not rely on DLLs that
aren't always installed on Windows. Notably this statically links the C
runtime (notably used by the C++ code) and means that the final binary
relies on fewer dlls.
This in theory means that the binaries are only limited by the number of
APIs they use from Windows. Note that this also matches how we build
releases of Rust for MSVC.
* Set MACOSX_DEPLOYMENT_TARGET for macOS releases
This is an effort to ideally produce "more portable" binaries for the
releases we publish to GitHub. Currently the way macOS works is that
you're generally only guaranteed to work on the same platform you built
on and later (although it may sometimes work on older platforms). By
configuring this environment variable it should be possible to lower the
binary compatibility requirement, allowing running binaries on older OS
releases than the build machine is running.
I've chosen 10.9 here since it seems to be the lowest that "just works",
but there's no particular reason other than that for choosing this. Rust
itself chooses 10.8 (I think) for the compiler and 10.7 for the standard
library. This decision is largely driven by the C++ code from wabt-sys
which has more requirements about binary compatibility than Rust code
does.
Note that I don't actually have older macOS machines to test on as well,
but I can at least confirm that this does affect the build process!
* Comment the env var added
* Refactor Azure Pipelines config and tweak releases
* Extract out doc/rustfmt jobs into their own separate builders. Helps
avoiding having to skip tons of steps and can get failing results more
quickly.
* Extract out Rust installation logic to a dedicated template.
* Separate out the build/test job matrices, where one matrix runs tests
and another runs a full build
* Refactor release directory structure to follow a convention where
`foo.tar.gz` extracts to a folder called `foo` and follow unix-like
conventions and copy over the license/readme files into the release
tarballs.
* Swap order of build/test
This Azure Pipelines setup compiles and tests Wasmtime for Linux, macOS, and Windows.
If the CI run was triggered by a new tag being created, a new release for that tag is created with a changelog relative to the last tag release and archives of the builds for all platforms.
If the CI run was triggered by new commits landing on `master`, the release `latest-master` is updated with a new changelog relative to the last tag release and archives of the new builds for all platforms.
Note: This PR also contains changes to disable a bunch of tests on Windows, which are failing due to issues with signal handling.
This change adds an `--enable-simd` flag to the binaries in this project. This allows the ISA `enable_simd` flag to be set and to configure the validation configuration used by wasmparser to allow SIMD instructions.
