* wiggle-runtime: add as_raw method for [T]
* add trivial borrow checker back in
* integrate runtime borrow checker with as_raw methods
* handle pointer arith overflow correctly in as_raw, create PtrOverflow error
* runtime: add validation back to GuestType
* generate: impl validate for enums, flags, handles, ints
* oops! make validate its own method on trait GuestTypeTransparent
* fix transparent impls for enum, flag, handle, int
* some structs are transparent. fix tests.
* tests: define byte_slice_strat and friends
* wiggle-tests: i believe my allocator is working now
* some type juggling around memset for ease of use
* make GuestTypeTransparent an unsafe trait
* delete redundant validation of pointer align
* fix doc
* wiggle_test: aha, you cant use sets to track memory areas
* add multi-string test
which exercises the runtime borrow checker against
HostMemory::byte_slice_strat
* oops left debug panic in
* remove redundant (& incorrect, since unchecked) length calc
* redesign validate again, and actually hook to as_raw
* makr all validate impls as inline
this should hopefully allow as_raw's check loop to be unrolled to a
no-op in most cases!
* code review fixes
This commit rewrites the runtime crate to provide safety in the face
of recursive calls to the guest. The basic principle is that
`GuestMemory` is now a trait which dynamically returns the
pointer/length pair. This also has an implicit contract (hence the
`unsafe` trait) that the pointer/length pair point to a valid list of
bytes in host memory "until something is reentrant".
After this changes the various suite of `Guest*` types were rewritten.
`GuestRef` and `GuestRefMut` were both removed since they cannot safely
exist. The `GuestPtrMut` type was removed for simplicity, and the final
`GuestPtr` type subsumes `GuestString` and `GuestArray`. This means
that there's only one guest pointer type, `GuestPtr<'a, T>`, where `'a`
is the borrow into host memory, basically borrowing the `GuestMemory`
trait object itself.
Some core utilities are exposed on `GuestPtr`, but they're all 100%
safe. Unsafety is now entirely contained within a few small locations:
* Implementations of the `GuestType` for primitive types (e.g. `i8`,
`u8`, etc) use `unsafe` to read/write memory. The `unsafe` trait of
`GuestMemory` though should prove that they're safe.
* `GuestPtr<'_, str>` has a method which validates utf-8 contents, and
this requires `unsafe` internally to read all the bytes. This is
guaranteed to be safe however given the contract of `GuestMemory`.
And that's it! Everything else is a bunch of safe combinators all built
up on the various utilities provided by `GuestPtr`. The general idioms
are roughly the same as before, with various tweaks here and there. A
summary of expected idioms are:
* For small values you'd `.read()` or `.write()` very quickly. You'd
pass around the type itself.
* For strings, you'd pass `GuestPtr<'_, str>` down to the point where
it's actually consumed. At that moment you'd either decide to copy it
out (a safe operation) or you'd get a raw view to the string (an
unsafe operation) and assert that you won't call back into wasm while
you're holding that pointer.
* Arrays are similar to strings, passing around `GuestPtr<'_, [T]>`.
Arrays also have a `iter()` method which yields an iterator of
`GuestPtr<'_, T>` for convenience.
Overall there's still a lot of missing documentation on the runtime
crate specifically around the safety of the `GuestMemory` trait as well
as how the utilities/methods are expected to be used. Additionally
there's utilities which aren't currently implemented which would be easy
to implement. For example there's no method to copy out a string or a
slice, although that would be pretty easy to add.
In any case I'm curious to get feedback on this approach and see what
y'all think!
This commit refactors trait system for guest types. Namely, as
discussed offline on zulip, `GuestType` now includes `GuestTypeClone`,
whereas `GuestTypeCopy` has been renamed to `GuestTypeTransparent`.
* Add basic GuestString support to wiggle
This commit adds basic `GuestString` support to `wiggle`. `GuestString`
is a wrapper around `GuestArray<'_, u8>` array type which itself can
be made into either an owned (cloned) Rust `String` or borrowed as
a reference `&str`. In both cases, `GuestString` ensures that the
underlying bytes are valid Unicode code units, throwing a `InvalidUtf8`
error if not.
This commit adds support *only* for passing in strings as arguments
in WASI. Marshalling of the return arg has not yet been implemented.
I'm not even sure it's possible without multi-value return args
feature of Wasm. It's not a major setback especially since the WASI
spec (and this includes even the `ephemeral` snapshot) doesn't
return strings anywhere. They are only ever passed in as arguments
to interface functions.
It should be noted that error returned in case of invalid UTF-8
requires a lot more love as it doesn't include anything besides
flagging an event that the string contained an invalid Unicode code unit.
* Borrow all of string's memory including nul-byte
Borrow all of string's underlying memory including the nul-byte.
This perhaps might not have a tremendous impact on anything, but
since the nul-byte is technically part of the WASI string, we should
include it in the borrow as well.
* Fill in wiggle-generate blanks for strings
* Print to screen passed string in proptest
* Strings are PointerLengthPairs!
* Fix generation of strings in compound types
* Update test with simple string strategy
* Generate better test strings
* Finalise proptest for strings
* Fix formatting
* Update crates/runtime/src/memory/string.rs
Removes unnecessary comment in code
* Apply Pat's suggestion to wrap Utf8Error as error
* merge GuestTypePtr and GuestTypeClone into a single GuestTypeClone<'a> trait
* GuestArray can derive Clone (but not impl GuestTypeClone)
* fix array tests
* Fix GuestTypeClone for flags
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
* Add array generation to wiggle-generate crate
This commit:
* adds array generation to `wiggle-generate` crate which implies
that we can now generate arrays from `witx` files
* introduces two test interface functions `foo::reduce_excuses` and
`foo::populate_excuses`, and adds matching prop-tests
* adds an out-of-boundary check to `HostMemory::mem_area_strat` since
now, given we're generating arrays for testing with an arbitrary
but bounded number of elements, it is possible to violate the boundary
* refactors `Region::extend` to a new signature `extend(times: u32)`
which multiplies the current pointer `len` by `times`
* fixes bug in `GuestArray::as_ref` and `GuestArrayMut::as_ref_mut` methods
where we were not validating the first element (we always started the
validation from the second element)
* Fix generation of arrays in witx
This commit fixes how `arrays` are auto-generated from `witx` files.
In particular, the changes include:
* Since by design every `array` in `witx` represents an immutable
slab of memory, we will only ever operate on `GuestArray` in which
case I've gone ahead and removed `GuestArrayMut` so as to unclutter
the code somewhat. If we find ourselves in need for it in the future,
I reckon it will be better to write it from scratch (since the codebase
will inevitably evolve by then considerably) rather than maintaining an
unused implementation.
* I've rewritten `GuestArrayRef<T>` to be a wrapper for `Vec<GuestRef<T>>`.
Also, `GuestArray::as_ref` now borrows each underlying "element" of the
array one-by-one rather than borrowing the entire chunk of memory at once.
This change is motivated by the inability to coerce type parameter `T` in
`GuestArray<T>` in more complicated cases such as arrays of guest pointers
`GuestPtr<T>` to `*const T` for reuse in `std::slice::from_raw_parts` call.
(In general, the size of Wasm32 pointer is 4 bytes, while
```
std::mem::size_of::<T>() == std::mem::size_of::<GuestPtr<S>>() == 16
```
which is problematic; i.e., I can't see how I could properly extract guest
pointers from slices of 4 bytes and at the same time not allocate.)
* I've augmented fuzz tests by (re-)defining two `array` types:
```
(typename $const_excuse_array (array (@witx const_pointer $excuse)))
(typename $excuse_array (array (@witx pointer $excuse)))
```
This should hopefully emulate and test the `iovec` and `ciovec` arrays
present in WASI spec.
* Refactor naming and crates info
This commit:
* changes workspace crates to have a `wiggle_` prefix in names
* rename `memory` module of `wiggle-memory` crate to `runtime`
* fixes authors of all crates
* Rename wiggle memory crate to runtime
