Add heaps to the Cretonne IL.

Add preamble syntax for declaring static and dynamic heaps, and update
the langref section on heaps. Add IR support for heap references.

Remove the heap_load and heap_store as discussed in #144. We will use
heap_addr along with native load and store instructions in their place.

Add the heap_addr instruction and document its bounds checking
semantics.

cranelift/docs/example.cton

@@ -13,7 +13,7 @@ ebb1(v1: i32, v2: i32):
 ebb2(v5: i32):
     v6 = imul_imm v5, 4
     v7 = iadd v1, v6
-    v8 = heap_load.f32 v7         ; array[i]
+    v8 = load.f32 v7              ; array[i]
     v9 = fpromote.f64 v8
     v10 = stack_load.f64 ss1
     v11 = fadd v9, v10

cranelift/docs/langref.rst

@@ -559,44 +559,82 @@ all process memory. Instead, it is given a small set of memory areas to work
 in, and all accesses are bounds checked. Cretonne models this through the
 concept of *heaps*.
 
-A heap is declared in the function preamble and can be accessed with restricted
+A heap is declared in the function preamble and can be accessed with the
-instructions that trap on out-of-bounds accesses. Heap addresses can be smaller
+:inst:`heap_addr` instruction that traps on out-of-bounds accesses or returns a
-than the native pointer size, for example unsigned :type:`i32` offsets on a
+pointer that is guaranteed to trap. Heap addresses can be smaller than the
-64-bit architecture.
+native pointer size, for example unsigned :type:`i32` offsets on a 64-bit
+architecture.
 
-.. inst:: H = heap Name
+.. digraph:: static
+    :align: center
+    :caption: Heap address space layout
 
-    Declare a heap in the function preamble.
+    node [
+        shape=record,
+        fontsize=10,
+        fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans"
+    ]
+    "static" [label="mapped\npages|unmapped\npages|guard\npages"]
 
-    This doesn't allocate memory, it just retrieves a handle to a sandbox from
+A heap appears as three consecutive ranges of address space:
-    the runtime environment.
 
-    :arg Name: String identifying the heap in the runtime environment.
+1. The *mapped pages* are the usable memory range in the heap. Loads and stores
-    :result H: Heap identifier.
+   to this range won't trap. A heap may have a minimum guaranteed size which
+   means that some mapped pages are always present.
+2. The *unmapped pages* is a possibly empty range of address space that may be
+   mapped in the future when the heap is grown.
+3. The *guard pages* is a range of address space that is guaranteed to cause a
+   trap when accessed. It is used to optimize bounds checking for heap accesses
+   with a shared base pointer.
 
-.. autoinst:: heap_load
+The *heap bound* is the total size of the mapped and unmapped pages. This is
-.. autoinst:: heap_store
+the bound that :inst:`heap_addr` checks against. Memory accesses inside the
-
+heap bounds can trap if they hit an unmapped page.
-When optimizing heap accesses, Cretonne may separate the heap bounds checking
-and address computations from the memory accesses.
 
 .. autoinst:: heap_addr
 
-A small example using heaps::
+Two styles of heaps are supported, *static* and *dynamic*. They behave
+differently when resized.
 
-    function %vdup(i32, i32) {
+Static heaps
-        h1 = heap "main"
+~~~~~~~~~~~~
 
-    ebb1(v1: i32, v2: i32):
+A *static heap* starts out with all the address space it will ever need, so it
-        v3 = heap_load.i32x4 h1, v1, 0
+never moves to a different address. At the base address is a number of mapped
-        v4 = heap_addr h1, v2, 32      ; Shared range check for two stores.
+pages corresponding to the heap's current size. Then follows a number of
-        store v3, v4, 0
+unmapped pages where the heap can grow up to its maximum size. After the
-        store v3, v4, 16
+unmapped pages follow the guard pages which are also guaranteed to generate a
-        return
+trap when accessed.
-    }
 
-The final expansion of the :inst:`heap_addr` range check and address conversion
+.. inst:: H = static Base, min MinBytes, bound BoundBytes, guard GuardBytes
-depends on the runtime environment.
+
+    Declare a static heap in the preamble.
+
+    :arg Base: Global variable holding the heap's base address or
+            ``reserved_reg``.
+    :arg MinBytes: Guaranteed minimum heap size in bytes. Accesses below this
+            size will never trap.
+    :arg BoundBytes: Fixed heap bound in bytes. This defines the amount of
+            address space reserved for the heap, not including the guard pages.
+    :arg GuardBytes: Size of the guard pages in bytes.
+
+Dynamic heaps
+~~~~~~~~~~~~~
+
+A *dynamic heap* can be relocated to a different base address when it is
+resized, and its bound can move dynamically. The guard pages move when the heap
+is resized. The bound of a dynamic heap is stored in a global variable.
+
+.. inst:: H = dynamic Base, min MinBytes, bound BoundGV, guard GuardBytes
+
+    Declare a dynamic heap in the preamble.
+
+    :arg Base: Global variable holding the heap's base address or
+            ``reserved_reg``.
+    :arg MinBytes: Guaranteed minimum heap size in bytes. Accesses below this
+            size will never trap.
+    :arg BoundGV: Global variable containing the current heap bound in bytes.
+    :arg GuardBytes: Size of the guard pages in bytes.
 
 
 Operations

cranelift/filetests/parser/memory.cton

@@ -35,3 +35,32 @@ ebb0:
     v1 = global_addr.i32 gv1
     return v1
 }
+
+; Declare static heaps.
+function %sheap(i32) -> i64 {
+    heap1 = static reserved_reg, min 0x1_0000, bound 0x1_0000_0000, guard 0x8000_0000
+    heap2 = static gv5, guard 0x1000, bound 0x1_0000
+    gv5 = vmctx+64
+
+    ; check: $heap1 = static reserved_reg, min 0x0001_0000, bound 0x0001_0000_0000, guard 0x8000_0000
+    ; check: $heap2 = static $gv5, min 0, bound 0x0001_0000, guard 4096
+ebb0(v1: i32):
+    v2 = heap_addr.i64 heap1, v1, 0
+    ; check: $v2 = heap_addr.i64 $heap1, $v1, 0
+    return v2
+}
+
+; Declare dynamic heaps.
+function %dheap(i32) -> i64 {
+    heap1 = dynamic reserved_reg, min 0x1_0000, bound gv6, guard 0x8000_0000
+    heap2 = dynamic gv5, bound gv6, guard 0x1000
+    gv5 = vmctx+64
+    gv6 = vmctx+72
+
+    ; check: $heap1 = dynamic reserved_reg, min 0x0001_0000, bound $gv6, guard 0x8000_0000
+    ; check: $heap2 = dynamic $gv5, min 0, bound $gv6, guard 4096
+ebb0(v1: i32):
+    v2 = heap_addr.i64 heap2, v1, 0
+    ; check: $v2 = heap_addr.i64 $heap2, $v1, 0
+    return v2
+}

cranelift/filetests/parser/tiny.cton

@@ -136,20 +136,6 @@ ebb0:
 ; nextln: stack_store $v1, $ss10+2
 ; nextln: stack_store $v2, $ss2
 
-; Heap access instructions.
-function %heap(i32) {
-    ; TODO: heap0 = heap %foo
-ebb0(v1: i32):
-    v2 = heap_load.f32 v1
-    v3 = heap_load.f32 v1+12
-    heap_store v3, v1
-}
-; sameln: function %heap(i32) native {
-; nextln: ebb0($v1: i32):
-; nextln:     $v2 = heap_load.f32 $v1
-; nextln:     $v3 = heap_load.f32 $v1+12
-; nextln:     heap_store $v3, $v1
-
 ; Memory access instructions.
 function %memory(i32) {
 ebb0(v1: i32):

lib/cretonne/meta/base/entities.py

@@ -30,3 +30,6 @@ func_ref = EntityRefKind('func_ref', 'An external function.')
 #: A reference to a jump table declared in the function preamble.
 jump_table = EntityRefKind(
         'jump_table', 'A jump table.', default_member='table')
+
+#: A reference to a heap declared in the function preamble.
+heap = EntityRefKind('heap', 'A heap.')

lib/cretonne/meta/base/formats.py

@@ -8,10 +8,11 @@ in this module.
 from __future__ import absolute_import
 from cdsl.formats import InstructionFormat
 from cdsl.operands import VALUE, VARIABLE_ARGS
-from .immediates import imm64, uimm8, ieee32, ieee64, offset32, uoffset32
+from .immediates import imm64, uimm8, uimm32, ieee32, ieee64
+from .immediates import offset32, uoffset32
 from .immediates import boolean, intcc, floatcc, memflags, regunit
 from . import entities
-from .entities import ebb, sig_ref, func_ref, stack_slot
+from .entities import ebb, sig_ref, func_ref, stack_slot, heap
 
 Nullary = InstructionFormat()
 
@@ -59,6 +60,7 @@ StackStore = InstructionFormat(VALUE, stack_slot, offset32)
 # TODO: Add a reference to a `heap` declared in the preamble.
 HeapLoad = InstructionFormat(VALUE, uoffset32)
 HeapStore = InstructionFormat(VALUE, VALUE, uoffset32)
+HeapAddr = InstructionFormat(heap, VALUE, uimm32)
 
 RegMove = InstructionFormat(VALUE, ('src', regunit), ('dst', regunit))
 

lib/cretonne/meta/base/instructions.py

@@ -9,7 +9,7 @@ from cdsl.operands import Operand, VARIABLE_ARGS
 from cdsl.typevar import TypeVar
 from cdsl.instructions import Instruction, InstructionGroup
 from base.types import f32, f64, b1
-from base.immediates import imm64, uimm8, ieee32, ieee64, offset32, uoffset32
+from base.immediates import imm64, uimm8, uimm32, ieee32, ieee64, offset32
 from base.immediates import boolean, intcc, floatcc, memflags, regunit
 from base import entities
 from cdsl.ti import WiderOrEq
@@ -360,40 +360,26 @@ global_addr = Instruction(
 #
 # WebAssembly bounds-checked heap accesses.
 #
-# TODO: Add a `heap` operand that selects between multiple heaps.
-# TODO: Should the immediate offset be a `u32`?
-# TODO: Distinguish between `iAddr` for a heap and for a target address? i.e.,
-#       32-bit WebAssembly on a 64-bit target has two different types.
 
-Offset = Operand('Offset', uoffset32, 'Unsigned offset to effective address')
+HeapOffset = TypeVar('HeapOffset', 'An unsigned heap offset', ints=(32, 64))
 
-heap_load = Instruction(
+H = Operand('H', entities.heap)
-        'heap_load', r"""
+p = Operand('p', HeapOffset)
-        Load a value at the address :math:`p + Offset` in the heap H.
+Size = Operand('Size', uimm32, 'Size in bytes')
-
-        Trap if the heap access would be out of bounds.
-        """,
-        ins=(p, Offset), outs=a, can_load=True)
-
-heap_store = Instruction(
-        'heap_store', r"""
-        Store a value at the address :math:`p + Offset` in the heap H.
-
-        Trap if the heap access would be out of bounds.
-        """,
-        ins=(x, p, Offset), can_store=True)
 
 heap_addr = Instruction(
         'heap_addr', r"""
         Bounds check and compute absolute address of heap memory.
 
-        Verify that the address range ``p .. p + Size - 1`` is valid in the
+        Verify that the offset range ``p .. p + Size - 1`` is in bounds for the
-        heap H, and trap if not.
+        heap H, and generate an absolute address that is safe to dereference.
 
-        Convert the heap-relative address in ``p`` to a real absolute address
+        1. If ``p + Size`` is not greater than the heap bound, return an
-        and return it.
+           absolute address corresponding to a byte offset of ``p`` from the
+           heap's base address.
+        2. If ``p + Size`` is greater than the heap bound, generate a trap.
         """,
-        ins=(p, Offset), outs=addr)
+        ins=(H, p, Size), outs=addr)
 
 #
 # Materializing constants.

lib/cretonne/src/ir/entities.rs

@@ -98,6 +98,11 @@ entity_impl!(FuncRef, "fn");
 pub struct SigRef(u32);
 entity_impl!(SigRef, "sig");
 
+/// A reference to a heap.
+#[derive(Copy, Clone, PartialEq, Eq, Hash, Debug)]
+pub struct Heap(u32);
+entity_impl!(Heap, "heap");
+
 /// A reference to any of the entities defined in this module.
 #[derive(Copy, Clone, PartialEq, Eq, Hash, Debug)]
 pub enum AnyEntity {
@@ -119,6 +124,8 @@ pub enum AnyEntity {
     FuncRef(FuncRef),
     /// A function call signature.
     SigRef(SigRef),
+    /// A heap.
+    Heap(Heap),
 }
 
 impl fmt::Display for AnyEntity {
@@ -133,6 +140,7 @@ impl fmt::Display for AnyEntity {
             AnyEntity::JumpTable(r) => r.fmt(f),
             AnyEntity::FuncRef(r) => r.fmt(f),
             AnyEntity::SigRef(r) => r.fmt(f),
+            AnyEntity::Heap(r) => r.fmt(f),
         }
     }
 }
@@ -185,6 +193,12 @@ impl From<SigRef> for AnyEntity {
     }
 }
 
+impl From<Heap> for AnyEntity {
+    fn from(r: Heap) -> AnyEntity {
+        AnyEntity::Heap(r)
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;

lib/cretonne/src/ir/function.rs

@@ -29,6 +29,9 @@ pub struct Function {
     /// Global variables referenced.
     pub global_vars: PrimaryMap<ir::GlobalVar, ir::GlobalVarData>,
 
+    /// Heaps referenced.
+    pub heaps: PrimaryMap<ir::Heap, ir::HeapData>,
+
     /// Jump tables used in this function.
     pub jump_tables: JumpTables,
 
@@ -61,6 +64,7 @@ impl Function {
             signature: sig,
             stack_slots: StackSlots::new(),
             global_vars: PrimaryMap::new(),
+            heaps: PrimaryMap::new(),
             jump_tables: PrimaryMap::new(),
             dfg: DataFlowGraph::new(),
             layout: Layout::new(),

lib/cretonne/src/ir/heap.rs

@@ -0,0 +1,70 @@
+//! Heaps.
+
+use ir::immediates::Imm64;
+use ir::GlobalVar;
+use std::fmt;
+
+/// Information about a heap declaration.
+#[derive(Clone)]
+pub struct HeapData {
+    /// Method for determining the heap base address.
+    pub base: HeapBase,
+
+    /// Guaranteed minimum heap size in bytes. Heap accesses before `min_size` don't need bounds
+    /// checking.
+    pub min_size: Imm64,
+
+    /// Size in bytes of the guard pages following the heap.
+    pub guard_size: Imm64,
+
+    /// Heap style, with additional style-specific info.
+    pub style: HeapStyle,
+}
+
+/// Method for determining the base address of a heap.
+#[derive(Clone)]
+pub enum HeapBase {
+    /// The heap base lives in a reserved register.
+    ReservedReg,
+
+    /// The heap base is in a global variable.
+    GlobalVar(GlobalVar),
+}
+
+/// Style of heap including style-specific information.
+#[derive(Clone)]
+pub enum HeapStyle {
+    /// A dynamic heap can be relocated to a different base address when it is grown.
+    Dynamic {
+        /// Global variable holding the current bound of the heap in bytes.
+        bound_gv: GlobalVar,
+    },
+
+    /// A static heap has a fixed base address and a number of not-yet-allocated pages before the
+    /// guard pages.
+    Static {
+        /// Heap bound in bytes. The guard pages are allocated after the bound.
+        bound: Imm64,
+    },
+}
+
+impl fmt::Display for HeapData {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        f.write_str(match self.style {
+                           HeapStyle::Dynamic { .. } => "dynamic",
+                           HeapStyle::Static { .. } => "static",
+                       })?;
+
+        match self.base {
+            HeapBase::ReservedReg => write!(f, " reserved_reg")?,
+            HeapBase::GlobalVar(gv) => write!(f, " {}", gv)?,
+        }
+
+        write!(f, ", min {}", self.min_size)?;
+        match self.style {
+            HeapStyle::Dynamic { bound_gv } => write!(f, ", bound {}", bound_gv)?,
+            HeapStyle::Static { bound } => write!(f, ", bound {}", bound)?,
+        }
+        write!(f, ", guard {}", self.guard_size)
+    }
+}

lib/cretonne/src/ir/instructions.rs

@@ -12,7 +12,7 @@ use std::ops::{Deref, DerefMut};
 
 use ir;
 use ir::{Value, Type, Ebb, JumpTable, SigRef, FuncRef, StackSlot, MemFlags};
-use ir::immediates::{Imm64, Uimm8, Ieee32, Ieee64, Offset32, Uoffset32};
+use ir::immediates::{Imm64, Uimm8, Uimm32, Ieee32, Ieee64, Offset32, Uoffset32};
 use ir::condcodes::*;
 use ir::types;
 use isa::RegUnit;
@@ -199,6 +199,12 @@ pub enum InstructionData {
         args: [Value; 2],
         offset: Uoffset32,
     },
+    HeapAddr {
+        opcode: Opcode,
+        heap: ir::Heap,
+        arg: Value,
+        imm: Uimm32,
+    },
     Load {
         opcode: Opcode,
         flags: MemFlags,

lib/cretonne/src/ir/mod.rs

@@ -14,18 +14,20 @@ mod builder;
 mod extfunc;
 mod funcname;
 mod globalvar;
+mod heap;
 mod memflags;
 mod progpoint;
 mod valueloc;
 
 pub use ir::builder::{InstBuilder, InstBuilderBase, InstInserterBase, InsertBuilder};
 pub use ir::dfg::{DataFlowGraph, ValueDef};
-pub use ir::entities::{Ebb, Inst, Value, StackSlot, GlobalVar, JumpTable, FuncRef, SigRef};
+pub use ir::entities::{Ebb, Inst, Value, StackSlot, GlobalVar, JumpTable, FuncRef, SigRef, Heap};
 pub use ir::extfunc::{Signature, CallConv, ArgumentType, ArgumentExtension, ArgumentPurpose,
                       ExtFuncData};
 pub use ir::funcname::FunctionName;
 pub use ir::function::Function;
 pub use ir::globalvar::GlobalVarData;
+pub use ir::heap::{HeapData, HeapStyle, HeapBase};
 pub use ir::instructions::{Opcode, InstructionData, VariableArgs, ValueList, ValueListPool};
 pub use ir::jumptable::JumpTableData;
 pub use ir::layout::{Layout, CursorBase, Cursor};

lib/cretonne/src/verifier/mod.rs

@@ -301,6 +301,9 @@ impl<'a> Verifier<'a> {
             UnaryGlobalVar { global_var, .. } => {
                 self.verify_global_var(inst, global_var)?;
             }
+            HeapAddr { heap, .. } => {
+                self.verify_heap(inst, heap)?;
+            }
 
             // Exhaustive list so we can't forget to add new formats
             Nullary { .. } |
@@ -367,6 +370,14 @@ impl<'a> Verifier<'a> {
         }
     }
 
+    fn verify_heap(&self, inst: Inst, heap: ir::Heap) -> Result {
+        if !self.func.heaps.is_valid(heap) {
+            err!(inst, "invalid heap {}", heap)
+        } else {
+            Ok(())
+        }
+    }
+
     fn verify_value_list(&self, inst: Inst, l: &ValueList) -> Result {
         if !l.is_valid(&self.func.dfg.value_lists) {
             err!(inst, "invalid value list reference {:?}", l)

lib/cretonne/src/write.rs

@@ -54,6 +54,11 @@ fn write_preamble(w: &mut Write,
         writeln!(w, "    {} = {}", gv, func.global_vars[gv])?;
     }
 
+    for heap in func.heaps.keys() {
+        any = true;
+        writeln!(w, "    {} = {}", heap, func.heaps[heap])?;
+    }
+
     // Write out all signatures before functions since function declarations can refer to
     // signatures.
     for sig in func.dfg.signatures.keys() {
@@ -325,6 +330,7 @@ pub fn write_operands(w: &mut Write,
         } => write!(w, " {}, {}{}", arg, stack_slot, offset),
         HeapLoad { arg, offset, .. } => write!(w, " {}{}", arg, offset),
         HeapStore { args, offset, .. } => write!(w, " {}, {}{}", args[0], args[1], offset),
+        HeapAddr { heap, arg, imm, .. } => write!(w, " {}, {}, {}", heap, arg, imm),
         Load { flags, arg, offset, .. } => write!(w, "{} {}{}", flags, arg, offset),
         Store {
             flags,

lib/reader/src/lexer.rs

@@ -38,6 +38,7 @@ pub enum Token<'a> {
     Ebb(Ebb), // ebb3
     StackSlot(u32), // ss3
     GlobalVar(u32), // gv3
+    Heap(u32), // heap2
     JumpTable(u32), // jt2
     FuncRef(u32), // fn2
     SigRef(u32), // sig2
@@ -310,6 +311,7 @@ impl<'a> Lexer<'a> {
             "ebb" => Ebb::with_number(number).map(Token::Ebb),
             "ss" => Some(Token::StackSlot(number)),
             "gv" => Some(Token::GlobalVar(number)),
+            "heap" => Some(Token::Heap(number)),
             "jt" => Some(Token::JumpTable(number)),
             "fn" => Some(Token::FuncRef(number)),
             "sig" => Some(Token::SigRef(number)),

lib/reader/src/parser.rs

@@ -12,9 +12,9 @@ use std::mem;
 use cretonne::ir::{Function, Ebb, Opcode, Value, Type, FunctionName, CallConv, StackSlotData,
                    JumpTable, JumpTableData, Signature, ArgumentType, ArgumentExtension,
                    ExtFuncData, SigRef, FuncRef, StackSlot, ValueLoc, ArgumentLoc, MemFlags,
-                   GlobalVar, GlobalVarData};
+                   GlobalVar, GlobalVarData, Heap, HeapData, HeapStyle, HeapBase};
 use cretonne::ir::types::VOID;
-use cretonne::ir::immediates::{Imm64, Offset32, Uoffset32, Ieee32, Ieee64};
+use cretonne::ir::immediates::{Imm64, Uimm32, Offset32, Uoffset32, Ieee32, Ieee64};
 use cretonne::ir::entities::AnyEntity;
 use cretonne::ir::instructions::{InstructionFormat, InstructionData, VariableArgs};
 use cretonne::isa::{self, TargetIsa, Encoding, RegUnit};
@@ -138,7 +138,21 @@ impl<'a> Context<'a> {
     fn get_gv(&self, number: u32, loc: &Location) -> Result<GlobalVar> {
         match self.map.get_gv(number) {
             Some(gv) => Ok(gv),
-            None => err!(loc, "undefined stack slot ss{}", number),
+            None => err!(loc, "undefined global variable gv{}", number),
+        }
+    }
+
+    // Allocate a heap slot and add a mapping number -> Heap.
+    fn add_heap(&mut self, number: u32, data: HeapData, loc: &Location) -> Result<()> {
+        self.map
+            .def_heap(number, self.function.heaps.push(data), loc)
+    }
+
+    // Resolve a reference to a heap.
+    fn get_heap(&self, number: u32, loc: &Location) -> Result<Heap> {
+        match self.map.get_heap(number) {
+            Some(heap) => Ok(heap),
+            None => err!(loc, "undefined heap heap{}", number),
         }
     }
 
@@ -255,6 +269,23 @@ impl<'a> Context<'a> {
             }
         }
 
+        // Rewrite references to global variables in heaps.
+        for heap in self.function.heaps.keys() {
+            let loc = heap.into();
+            match self.function.heaps[heap].base {
+                HeapBase::GlobalVar(ref mut base) => {
+                    self.map.rewrite_gv(base, loc)?;
+                }
+                _ => {}
+            }
+            match self.function.heaps[heap].style {
+                HeapStyle::Dynamic { ref mut bound_gv } => {
+                    self.map.rewrite_gv(bound_gv, loc)?;
+                }
+                _ => {}
+            }
+        }
+
         Ok(())
     }
 }
@@ -381,6 +412,17 @@ impl<'a> Parser<'a> {
         }
     }
 
+    // Match and consume a global variable reference in the preamble where it can't be rewritten.
+    //
+    // Any global variable references appearing in the preamble need to be rewritten after parsing
+    // the whole preamble.
+    fn match_gv_preamble(&mut self, err_msg: &str) -> Result<GlobalVar> {
+        match GlobalVar::with_number(self.match_gv(err_msg)?) {
+            Some(gv) => Ok(gv),
+            None => err!(self.loc, "Invalid global variable number"),
+        }
+    }
+
     // Match and consume a function reference.
     fn match_fn(&mut self, err_msg: &str) -> Result<u32> {
         if let Some(Token::FuncRef(fnref)) = self.token() {
@@ -401,6 +443,16 @@ impl<'a> Parser<'a> {
         }
     }
 
+    // Match and consume a heap reference.
+    fn match_heap(&mut self, err_msg: &str) -> Result<u32> {
+        if let Some(Token::Heap(heap)) = self.token() {
+            self.consume();
+            Ok(heap)
+        } else {
+            err!(self.loc, err_msg)
+        }
+    }
+
     // Match and consume a jump table reference.
     fn match_jt(&mut self) -> Result<u32> {
         if let Some(Token::JumpTable(jt)) = self.token() {
@@ -451,6 +503,18 @@ impl<'a> Parser<'a> {
         }
     }
 
+    // Match and consume a Uimm32 immediate.
+    fn match_uimm32(&mut self, err_msg: &str) -> Result<Uimm32> {
+        if let Some(Token::Integer(text)) = self.token() {
+            self.consume();
+            // Lexer just gives us raw text that looks like an integer.
+            // Parse it as an Uimm32 to check for overflow and other issues.
+            text.parse().map_err(|e| self.error(e))
+        } else {
+            err!(self.loc, err_msg)
+        }
+    }
+
     // Match and consume a u8 immediate.
     // This is used for lane numbers in SIMD vectors.
     fn match_uimm8(&mut self, err_msg: &str) -> Result<u8> {
@@ -945,6 +1009,11 @@ impl<'a> Parser<'a> {
                     self.parse_global_var_decl()
                         .and_then(|(num, dat)| ctx.add_gv(num, dat, &self.loc))
                 }
+                Some(Token::Heap(..)) => {
+                    self.gather_comments(ctx.function.heaps.next_key());
+                    self.parse_heap_decl()
+                        .and_then(|(num, dat)| ctx.add_heap(num, dat, &self.loc))
+                }
                 Some(Token::SigRef(..)) => {
                     self.gather_comments(ctx.function.dfg.signatures.next_key());
                     self.parse_signature_decl(ctx.unique_isa)
@@ -1018,13 +1087,7 @@ impl<'a> Parser<'a> {
             }
             "deref" => {
                 self.match_token(Token::LPar, "expected '(' in 'deref' global variable decl")?;
-                let base_num = self.match_gv("expected global variable: gv«n»")?;
+                let base = self.match_gv_preamble("expected global variable: gv«n»")?;
-                // The base global variable may not have been declared yet, so create a fake
-                // reference using the source number. We'll rewrite these later.
-                let base = match GlobalVar::with_number(base_num) {
-                    Some(gv) => gv,
-                    None => return err!(self.loc, "Invalid global variable number for deref base"),
-                };
                 self.match_token(Token::RPar, "expected ')' in 'deref' global variable decl")?;
                 let offset = self.optional_offset32()?;
                 GlobalVarData::Deref { base, offset }
@@ -1035,6 +1098,75 @@ impl<'a> Parser<'a> {
         Ok((number, data))
     }
 
+    // Parse a heap decl.
+    //
+    // heap-decl ::= * Heap(heap) "=" heap-desc
+    // heap-desc ::= heap-style heap-base { "," heap-attr }
+    // heap-style ::= "static" | "dynamic"
+    // heap-base ::= "reserved_reg"
+    //             | GlobalVar(base)
+    // heap-attr ::= "min" Imm64(bytes)
+    //             | "max" Imm64(bytes)
+    //             | "guard" Imm64(bytes)
+    //
+    fn parse_heap_decl(&mut self) -> Result<(u32, HeapData)> {
+        let number = self.match_heap("expected heap number: heap«n»")?;
+        self.match_token(Token::Equal, "expected '=' in heap declaration")?;
+
+        let style_name = self.match_any_identifier("expected 'static' or 'dynamic'")?;
+
+        // heap-desc ::= heap-style * heap-base { "," heap-attr }
+        // heap-base ::= * "reserved_reg"
+        //             | * GlobalVar(base)
+        let base = match self.token() {
+            Some(Token::Identifier("reserved_reg")) => HeapBase::ReservedReg,
+            Some(Token::GlobalVar(base_num)) => {
+                let base_gv = match GlobalVar::with_number(base_num) {
+                    Some(gv) => gv,
+                    None => return err!(self.loc, "invalid global variable number for heap base"),
+                };
+                HeapBase::GlobalVar(base_gv)
+            }
+            _ => return err!(self.loc, "expected heap base"),
+        };
+        self.consume();
+
+        let mut data = HeapData {
+            base,
+            min_size: 0.into(),
+            guard_size: 0.into(),
+            style: HeapStyle::Static { bound: 0.into() },
+        };
+
+        // heap-desc ::= heap-style heap-base * { "," heap-attr }
+        while self.optional(Token::Comma) {
+            match self.match_any_identifier("expected heap attribute name")? {
+                "min" => {
+                    data.min_size = self.match_imm64("expected integer min size")?;
+                }
+                "bound" => {
+                    data.style = match style_name {
+                        "dynamic" => {
+                            HeapStyle::Dynamic {
+                                bound_gv: self.match_gv_preamble("expected gv bound")?,
+                            }
+                        }
+                        "static" => {
+                            HeapStyle::Static { bound: self.match_imm64("expected integer bound")? }
+                        }
+                        t => return err!(self.loc, "unknown heap style '{}'", t),
+                    };
+                }
+                "guard" => {
+                    data.guard_size = self.match_imm64("expected integer guard size")?;
+                }
+                t => return err!(self.loc, "unknown heap attribute '{}'", t),
+            }
+        }
+
+        Ok((number, data))
+    }
+
     // Parse a signature decl.
     //
     // signature-decl ::= SigRef(sigref) "=" signature
@@ -1802,6 +1934,20 @@ impl<'a> Parser<'a> {
                     offset,
                 }
             }
+            InstructionFormat::HeapAddr => {
+                let heap = self.match_heap("expected heap identifier")
+                    .and_then(|h| ctx.get_heap(h, &self.loc))?;
+                self.match_token(Token::Comma, "expected ',' between operands")?;
+                let arg = self.match_value("expected SSA value heap address")?;
+                self.match_token(Token::Comma, "expected ',' between operands")?;
+                let imm = self.match_uimm32("expected 32-bit integer size")?;
+                InstructionData::HeapAddr {
+                    opcode,
+                    heap,
+                    arg,
+                    imm,
+                }
+            }
             InstructionFormat::Load => {
                 let flags = self.optional_memflags();
                 let addr = self.match_value("expected SSA value address")?;

lib/reader/src/sourcemap.rs

@@ -9,7 +9,7 @@
 
 use cretonne::entity::EntityRef;
 use cretonne::ir::entities::AnyEntity;
-use cretonne::ir::{StackSlot, GlobalVar, JumpTable, Ebb, Value, SigRef, FuncRef};
+use cretonne::ir::{StackSlot, GlobalVar, Heap, JumpTable, Ebb, Value, SigRef, FuncRef};
 use error::{Result, Location};
 use lexer::split_entity_name;
 use std::collections::HashMap;
@@ -21,6 +21,7 @@ pub struct SourceMap {
     ebbs: HashMap<Ebb, Ebb>, // ebbNN
     stack_slots: HashMap<u32, StackSlot>, // ssNN
     global_vars: HashMap<u32, GlobalVar>, // gvNN
+    heaps: HashMap<u32, Heap>, // heapNN
     signatures: HashMap<u32, SigRef>, // sigNN
     functions: HashMap<u32, FuncRef>, // fnNN
     jump_tables: HashMap<u32, JumpTable>, // jtNN
@@ -51,6 +52,11 @@ impl SourceMap {
         self.global_vars.get(&src_num).cloned()
     }
 
+    /// Look up a heap entity by its source number.
+    pub fn get_heap(&self, src_num: u32) -> Option<Heap> {
+        self.heaps.get(&src_num).cloned()
+    }
+
     /// Look up a signature entity by its source number.
     pub fn get_sig(&self, src_num: u32) -> Option<SigRef> {
         self.signatures.get(&src_num).cloned()
@@ -82,6 +88,7 @@ impl SourceMap {
                                              }
                                              "ss" => self.get_ss(num).map(AnyEntity::StackSlot),
                                              "gv" => self.get_gv(num).map(AnyEntity::GlobalVar),
+                                             "heap" => self.get_heap(num).map(AnyEntity::Heap),
                                              "sig" => self.get_sig(num).map(AnyEntity::SigRef),
                                              "fn" => self.get_fn(num).map(AnyEntity::FuncRef),
                                              "jt" => self.get_jt(num).map(AnyEntity::JumpTable),
@@ -161,6 +168,7 @@ pub trait MutableSourceMap {
     fn def_ebb(&mut self, src: Ebb, entity: Ebb, loc: &Location) -> Result<()>;
     fn def_ss(&mut self, src_num: u32, entity: StackSlot, loc: &Location) -> Result<()>;
     fn def_gv(&mut self, src_num: u32, entity: GlobalVar, loc: &Location) -> Result<()>;
+    fn def_heap(&mut self, src_num: u32, entity: Heap, loc: &Location) -> Result<()>;
     fn def_sig(&mut self, src_num: u32, entity: SigRef, loc: &Location) -> Result<()>;
     fn def_fn(&mut self, src_num: u32, entity: FuncRef, loc: &Location) -> Result<()>;
     fn def_jt(&mut self, src_num: u32, entity: JumpTable, loc: &Location) -> Result<()>;
@@ -177,6 +185,7 @@ impl MutableSourceMap for SourceMap {
             ebbs: HashMap::new(),
             stack_slots: HashMap::new(),
             global_vars: HashMap::new(),
+            heaps: HashMap::new(),
             signatures: HashMap::new(),
             functions: HashMap::new(),
             jump_tables: HashMap::new(),
@@ -216,6 +225,14 @@ impl MutableSourceMap for SourceMap {
         }
     }
 
+    fn def_heap(&mut self, src_num: u32, entity: Heap, loc: &Location) -> Result<()> {
+        if self.heaps.insert(src_num, entity).is_some() {
+            err!(loc, "duplicate heap: heap{}", src_num)
+        } else {
+            self.def_entity(entity.into(), loc)
+        }
+    }
+
     fn def_sig(&mut self, src_num: u32, entity: SigRef, loc: &Location) -> Result<()> {
         if self.signatures.insert(src_num, entity).is_some() {
             err!(loc, "duplicate signature: sig{}", src_num)