Reimplement the pooling instance allocation strategy (#5661)

* Reimplement the pooling instance allocation strategy

This commit is a reimplementation of the strategy by which the pooling
instance allocator selects a slot for a module. Previously there was a
choice amongst three different algorithms: "reuse affinity", "next
available", and "random". The default was "reuse affinity" but some new
data has come to light which shows that this may not always be a good
default.

Notably the pooling allocator will retain some memory per-slot in the
pooling instance allocator, for example instance data or memory data
if-so-configured. This means that a currently unused, but previously
used, slot can contribute to the RSS usage of a program using Wasmtime.
Consequently the RSS impact here is O(max slots) which can be
counter-intuitive for embedders. This particularly affects "reuse
affinity" because the algorithm for picking a slot when there are no
affine slots is "pick a random slot", which means eventually all slots
will get used.

In discussions about possible ways to tackle this, an alternative to
"pick a strategy" arose and is now implemented in this commit.
Concretely the new allocation algorithm for a slot is now:

* First pick the most recently used affine slot, if one exists.
* Otherwise if the number of affine slots to other modules is above some
  threshold N then pick the least-recently used affine slot.
* Otherwise pick a slot that's affine to nothing.

The "N" in this algorithm is configurable and setting it to 0 is the
same as the old "next available" strategy while setting it to infinity
is the same as the "reuse affinity" algorithm. Setting it to something
in the middle provides a knob to allow a modest "cache" of affine slots
while not allowing the total set of slots used to grow too much beyond
the maximal concurrent set of modules. The "random" strategy is now no
longer possible and was removed to help simplify the allocator.

* Resolve rustdoc warnings in `wasmtime-runtime` crate

* Remove `max_cold` as it duplicates the `slot_state.len()`

* More descriptive names

* Add a comment and debug assertion

* Add some list assertions

crates/runtime/src/cow.rs

@@ -305,7 +305,7 @@ impl ModuleMemoryImages {
 /// middle of it. Pictorially this data structure manages a virtual memory
 /// region that looks like:
 ///
-/// ```ignore
+/// ```text
 ///   +--------------------+-------------------+--------------+--------------+
 ///   |   anonymous        |      optional     |   anonymous  |    PROT_NONE |
 ///   |     zero           |       memory      |     zero     |     memory   |
@@ -333,7 +333,7 @@ impl ModuleMemoryImages {
 /// `accessible` limits are. Initially there is assumed to be no image in linear
 /// memory.
 ///
-/// When [`MemoryImageSlot::instantiate`] is called then the method will perform
+/// When `MemoryImageSlot::instantiate` is called then the method will perform
 /// a "synchronization" to take the image from its prior state to the new state
 /// for the image specified. The first instantiation for example will mmap the
 /// heap image into place. Upon reuse of a slot nothing happens except possibly
@@ -343,7 +343,7 @@ impl ModuleMemoryImages {
 /// A `MemoryImageSlot` is either `dirty` or it isn't. When a `MemoryImageSlot`
 /// is dirty then it is assumed that any memory beneath `self.accessible` could
 /// have any value. Instantiation cannot happen into a `dirty` slot, however, so
-/// the [`MemoryImageSlot::clear_and_remain_ready`] returns this memory back to
+/// the `MemoryImageSlot::clear_and_remain_ready` returns this memory back to
 /// its original state to mark `dirty = false`. This is done by resetting all
 /// anonymous memory back to zero and the image itself back to its initial
 /// contents.