components: Limit the recursive size of types in Wasmtime (#4825)

* components: Limit the recursive size of types in Wasmtime

This commit is aimed at fixing #4814 by placing a hard limit on the
maximal recursive depth a type may have in the component model. The
component model theoretically allows for infinite recursion but many
various types of operations within the component model are naturally
written as recursion over the structure of a type which can lead to
stack overflow with deeply recursive types. Some examples of recursive
operations are:

* Lifting and lowering a type - currently the recursion here is modeled
  in Rust directly with `#[derive]` implementations as well as the
  implementations for the `Val` type.

* Compilation of adapter trampolines which iterates over the type
  structure recursively.

* Historically many various calculations like the size of a type, the
  flattened representation of a type, etc, were all done recursively.
  Many of these are more efficiently done via other means but it was
  still natural to implement these recursively initially.

By placing a hard limit on type recursion Wasmtime won't be able to load
some otherwise-valid modules. The hope, though, is that no human-written
program is likely to ever reach this limit. This limit can be revised
and/or the locations with recursion revised if it's ever reached.

The implementation of this feature is done by generalizing the current
flattened-representation calculation which now keeps track of a type's
depth and size. The size calculation isn't used just yet but I plan to
use it in fixing #4816 and it was natural enough to write here as well.
The depth is checked after a type is translated and if it exceeds the
maximum then an error is returned.

Additionally the `Arbitrary for Type` implementation was updated to
prevent generation of a type that's too-recursive.

Closes #4814

* Remove unused size calculation

* Bump up just under the limit

crates/wasmtime/src/component/func/typed.rs

@@ -1114,7 +1114,7 @@ where
 
     fn typecheck(ty: &InterfaceType, types: &ComponentTypes) -> Result<()> {
         match ty {
-            InterfaceType::List(t) => T::typecheck(&types[*t], types),
+            InterfaceType::List(t) => T::typecheck(&types[*t].element, types),
             other => bail!("expected `list` found `{}`", desc(other)),
         }
     }
@@ -1322,10 +1322,7 @@ unsafe impl<T: ComponentType> ComponentType for WasmList<T> {
     const ABI: CanonicalAbiInfo = CanonicalAbiInfo::POINTER_PAIR;
 
     fn typecheck(ty: &InterfaceType, types: &ComponentTypes) -> Result<()> {
-        match ty {
-            InterfaceType::List(t) => T::typecheck(&types[*t], types),
-            other => bail!("expected `list` found `{}`", desc(other)),
-        }
+        <[T] as ComponentType>::typecheck(ty, types)
     }
 }
 

crates/wasmtime/src/component/types.rs

@@ -7,9 +7,9 @@ use std::mem;
 use std::ops::Deref;
 use std::sync::Arc;
 use wasmtime_environ::component::{
-    CanonicalAbiInfo, ComponentTypes, InterfaceType, TypeEnumIndex, TypeFlagsIndex,
-    TypeInterfaceIndex, TypeOptionIndex, TypeRecordIndex, TypeResultIndex, TypeTupleIndex,
-    TypeUnionIndex, TypeVariantIndex, VariantInfo,
+    CanonicalAbiInfo, ComponentTypes, InterfaceType, TypeEnumIndex, TypeFlagsIndex, TypeListIndex,
+    TypeOptionIndex, TypeRecordIndex, TypeResultIndex, TypeTupleIndex, TypeUnionIndex,
+    TypeVariantIndex, VariantInfo,
 };
 
 #[derive(Clone)]
@@ -39,7 +39,7 @@ impl<T: Eq> Eq for Handle<T> {}
 
 /// A `list` interface type
 #[derive(Clone, PartialEq, Eq, Debug)]
-pub struct List(Handle<TypeInterfaceIndex>);
+pub struct List(Handle<TypeListIndex>);
 
 impl List {
     /// Instantiate this type with the specified `values`.
@@ -49,7 +49,7 @@ impl List {
 
     /// Retreive the element type of this `list`.
     pub fn ty(&self) -> Type {
-        Type::from(&self.0.types[self.0.index], &self.0.types)
+        Type::from(&self.0.types[self.0.index].element, &self.0.types)
     }
 }