Reel in unsafety around InstanceHandle (#856)

* Reel in unsafety around `InstanceHandle`

This commit is an attempt, or at least is targeted at being a start, at
reeling in the unsafety around the `InstanceHandle` type. Currently this
type represents a sort of moral `Rc<Instance>` but is a bit more
specialized since the underlying memory is allocated through mmap.

Additionally, though, `InstanceHandle` exposes a fundamental flaw in its
safety by safetly allowing mutable access so long as you have `&mut
InstanceHandle`. This type, however, is trivially created by simply
cloning a `InstanceHandle` to get an owned reference. This means that
`&mut InstanceHandle` does not actually provide any guarantees about
uniqueness, so there's no more safety than `&InstanceHandle` itself.

This commit removes all `&mut self` APIs from `InstanceHandle`,
additionally removing some where `&self` was `unsafe` and `&mut self`
was safe (since it was trivial to subvert this "safety"). In doing so
interior mutability patterns are now used much more extensively through
structures such as `Table` and `Memory`. Additionally a number of
methods were refactored to be a bit clearer and use helper functions
where possible.

This is a relatively large commit unfortunately, but it snowballed very
quickly into touching quite a few places. My hope though is that this
will prevent developers working on wasmtime internals as well as
developers still yet to migrate to the `wasmtime` crate from falling
into trivial unsafe traps by accidentally using `&mut` when they can't.
All existing users relying on `&mut` will need to migrate to some form
of interior mutability, such as using `RefCell` or `Cell`.

This commit also additionally marks `InstanceHandle::new` as an `unsafe`
function. The rationale for this is that the `&mut`-safety is only the
beginning for the safety of `InstanceHandle`. In general the wasmtime
internals are extremely unsafe and haven't been audited for appropriate
usage of `unsafe`. Until that's done it's hoped that we can warn users
with this `unsafe` constructor and otherwise push users to the
`wasmtime` crate which we know is safe.

* Fix windows build

* Wrap up mutable memory state in one structure

Rather than having separate fields

* Use `Cell::set`, not `Cell::replace`, where possible

* Add a helper function for offsets from VMContext

* Fix a typo from merging

* rustfmt

* Use try_from, not as

* Tweak style of some setters

crates/wasi/src/instantiate.rs

@@ -2,6 +2,7 @@ use cranelift_codegen::ir::types;
 use cranelift_codegen::{ir, isa};
 use cranelift_entity::PrimaryMap;
 use cranelift_wasm::DefinedFuncIndex;
+use std::cell::RefCell;
 use std::fs::File;
 use std::sync::Arc;
 use target_lexicon::HOST;
@@ -78,15 +79,17 @@ pub fn instantiate_wasi_with_context(
     let data_initializers = Vec::new();
     let signatures = PrimaryMap::new();
 
-    InstanceHandle::new(
-        Arc::new(module),
-        finished_functions.into_boxed_slice(),
-        imports,
-        &data_initializers,
-        signatures.into_boxed_slice(),
-        None,
-        Box::new(wasi_ctx),
-    )
+    unsafe {
+        InstanceHandle::new(
+            Arc::new(module),
+            finished_functions.into_boxed_slice(),
+            imports,
+            &data_initializers,
+            signatures.into_boxed_slice(),
+            None,
+            Box::new(RefCell::new(wasi_ctx)),
+        )
+    }
 }
 
 wig::define_add_wrappers_to_module!(
@@ -94,11 +97,11 @@ wig::define_add_wrappers_to_module!(
 );
 
 // Used by `add_wrappers_to_module` defined in the macro above
-fn get_wasi_ctx(vmctx: &mut VMContext) -> Result<&mut WasiCtx, wasi::__wasi_errno_t> {
+fn get_wasi_ctx(vmctx: &mut VMContext) -> Result<&RefCell<WasiCtx>, wasi::__wasi_errno_t> {
     unsafe {
         vmctx
             .host_state()
-            .downcast_mut::<WasiCtx>()
+            .downcast_ref()
             .ok_or_else(|| panic!("no host state named WasiCtx available"))
     }
 }

crates/wasi/src/old/snapshot_0/instantiate.rs

@@ -2,6 +2,7 @@ use cranelift_codegen::ir::types;
 use cranelift_codegen::{ir, isa};
 use cranelift_entity::PrimaryMap;
 use cranelift_wasm::DefinedFuncIndex;
+use std::cell::RefCell;
 use std::fs::File;
 use std::sync::Arc;
 use target_lexicon::HOST;
@@ -78,23 +79,25 @@ pub fn instantiate_wasi_with_context(
     let data_initializers = Vec::new();
     let signatures = PrimaryMap::new();
 
-    InstanceHandle::new(
-        Arc::new(module),
-        finished_functions.into_boxed_slice(),
-        imports,
-        &data_initializers,
-        signatures.into_boxed_slice(),
-        None,
-        Box::new(wasi_ctx),
-    )
+    unsafe {
+        InstanceHandle::new(
+            Arc::new(module),
+            finished_functions.into_boxed_slice(),
+            imports,
+            &data_initializers,
+            signatures.into_boxed_slice(),
+            None,
+            Box::new(RefCell::new(wasi_ctx)),
+        )
+    }
 }
 
 // Used by `add_wrappers_to_module` defined in the macro above
-fn get_wasi_ctx(vmctx: &mut VMContext) -> Result<&mut WasiCtx, wasi::__wasi_errno_t> {
+fn get_wasi_ctx(vmctx: &mut VMContext) -> Result<&RefCell<WasiCtx>, wasi::__wasi_errno_t> {
     unsafe {
         vmctx
             .host_state()
-            .downcast_mut::<WasiCtx>()
+            .downcast_ref()
             .ok_or_else(|| panic!("no host state named WasiCtx available"))
     }
 }