Implement a setting for reserved dynamic memory growth (#3215)

* Implement a setting for reserved dynamic memory growth

Dynamic memories aren't really that heavily used in Wasmtime right now
because for most 32-bit memories they're classified as "static" which
means they reserve 4gb of address space and never move. Growth of a
static memory is simply making pages accessible, so it's quite fast.

With the memory64 feature, however, this is no longer true since all
memory64 memories are classified as "dynamic" at this time. Previous to
this commit growth of a dynamic memory unconditionally moved the entire
linear memory in the host's address space, always resulting in a new
`Mmap` allocation. This behavior is causing fuzzers to time out when
working with 64-bit memories because incrementally growing a memory by 1
page at a time can incur a quadratic time complexity as bytes are
constantly moved.

This commit implements a scheme where there is now a tunable setting for
memory to be reserved at the end of a dynamic memory to grow into. This
means that dynamic memory growth is ideally amortized as most calls to
`memory.grow` will be able to grow into the pre-reserved space. Some
calls, though, will still need to copy the memory around.

This helps enable a commented out test for 64-bit memories now that it's
fast enough to run in debug mode. This is because the growth of memory
in the test no longer needs to copy 4gb of zeros.

* Test fixes & review comments

* More comments

crates/wasmtime/src/config.rs

@@ -1166,6 +1166,45 @@ impl Config {
         self
     }
 
+    /// Configures the size, in bytes, of the extra virtual memory space
+    /// reserved after a "dynamic" memory for growing into.
+    ///
+    /// For the difference between static and dynamic memories, see the
+    /// [`Config::static_memory_maximum_size`]
+    ///
+    /// Dynamic memories can be relocated in the process's virtual address space
+    /// on growth and do not always reserve their entire space up-front. This
+    /// means that a growth of the memory may require movement in the address
+    /// space, which in the worst case can copy a large number of bytes from one
+    /// region to another.
+    ///
+    /// This setting configures how many bytes are reserved after the initial
+    /// reservation for a dynamic memory for growing into. A value of 0 here
+    /// means that no extra bytes are reserved and all calls to `memory.grow`
+    /// will need to relocate the wasm linear memory (copying all the bytes). A
+    /// value of 1 megabyte, however, means that `memory.grow` can allocate up
+    /// to a megabyte of extra memory before the memory needs to be moved in
+    /// linear memory.
+    ///
+    /// Note that this is a currently simple heuristic for optimizing the growth
+    /// of dynamic memories, primarily implemented for the memory64 propsal
+    /// where all memories are currently "dynamic". This is unlikely to be a
+    /// one-size-fits-all style approach and if you're an embedder running into
+    /// issues with dynamic memories and growth and are interested in having
+    /// other growth strategies available here please feel free to [open an
+    /// issue on the Wasmtime repository][issue]!
+    ///
+    /// [issue]: https://github.com/bytecodealliance/wasmtime/issues/ne
+    ///
+    /// ## Default
+    ///
+    /// For 64-bit platforms this defaults to 2GB, and for 32-bit platforms this
+    /// defaults to 1MB.
+    pub fn dynamic_memory_reserved_for_growth(&mut self, reserved: u64) -> &mut Self {
+        self.tunables.dynamic_memory_growth_reserve = round_up_to_pages(reserved);
+        self
+    }
+
     /// Indicates whether a guard region is present before allocations of
     /// linear memory.
     ///

crates/wasmtime/src/memory.rs

@@ -601,7 +601,7 @@ mod tests {
         let store = store.as_context();
         assert_eq!(store[mem.0].memory.offset_guard_size, 0);
         match &store[mem.0].memory.style {
-            wasmtime_environ::MemoryStyle::Dynamic => {}
+            wasmtime_environ::MemoryStyle::Dynamic { .. } => {}
             other => panic!("unexpected style {:?}", other),
         }
     }

crates/wasmtime/src/module/serialization.rs

@@ -429,6 +429,9 @@ impl<'a> SerializedModule<'a> {
             consume_fuel,
             static_memory_bound_is_maximum,
             guard_before_linear_memory,
+
+            // This doesn't affect compilation, it's just a runtime setting.
+            dynamic_memory_growth_reserve: _,
         } = self.tunables;
 
         Self::check_int(

crates/wasmtime/src/trampoline/memory.rs

@@ -63,7 +63,7 @@ impl RuntimeMemoryCreator for MemoryCreatorProxy {
             MemoryStyle::Static { bound } => {
                 Some(usize::try_from(bound * (WASM_PAGE_SIZE as u64)).unwrap())
             }
-            MemoryStyle::Dynamic => None,
+            MemoryStyle::Dynamic { .. } => None,
         };
         self.0
             .new_memory(