Initial draft of DSL semantics complete.

This latest refactor adds "extractor macros" in place of the
very-confusing-even-to-the-DSL-author reverse-rules-as-extractors
concept. It was beautifully symmetric but also just too mind-bending to
be practical.

It also adds argument polarity to external extractors. This is inspired
by Prolog's similar notion (see e.g. the "+x" vs. "-x" argument notation
in library documentation) where the unification-based semantics allow
for bidirectional flow through arguments. We don't want polymorphism
or dynamism w.r.t. directions/polarities here; the polarities are
static; but it is useful to be able to feed values *into* an extractor
(aside from the one value being extracted). Semantically this still
correlates to a term-rewriting/value-equivalence world since we can
still translate all of this to a list of equality constraints.

To make that work, this change also adds expressions into patterns,
specifically only for extractor "input" args. This required quite a bit
of internal refactoring but is only a small addition to the language
semantics.

I plan to build out the little instruction-selector sketch further but
the one that is here (in `test3.isle`) is starting to get interesting
already with the current DSL semantics.

cranelift/isle/src/ast.rs

@@ -12,6 +12,7 @@ pub struct Defs {
 pub enum Def {
     Type(Type),
     Rule(Rule),
+    Extractor(Extractor),
     Decl(Decl),
     Extern(Extern),
 }
@@ -69,6 +70,16 @@ pub struct Rule {
     pub prio: Option<i64>,
 }
 
+/// An extractor macro: (A x y) becomes (B x _ y ...). Expanded during
+/// ast-to-sema pass.
+#[derive(Clone, PartialEq, Eq, Debug)]
+pub struct Extractor {
+    pub term: Ident,
+    pub args: Vec<Ident>,
+    pub template: Pattern,
+    pub pos: Pos,
+}
+
 /// A pattern: the left-hand side of a rule.
 #[derive(Clone, PartialEq, Eq, Debug)]
 pub enum Pattern {
@@ -80,13 +91,40 @@ pub enum Pattern {
     /// An operator that matches a constant integer value.
     ConstInt { val: i64 },
     /// An application of a type variant or term.
-    Term { sym: Ident, args: Vec<Pattern> },
+    Term {
+        sym: Ident,
+        args: Vec<TermArgPattern>,
+    },
     /// An operator that matches anything.
     Wildcard,
     /// N sub-patterns that must all match.
     And { subpats: Vec<Pattern> },
 }
 
+impl Pattern {
+    pub fn root_term(&self) -> Option<&Ident> {
+        match self {
+            &Pattern::BindPattern { ref subpat, .. } => subpat.root_term(),
+            &Pattern::Term { ref sym, .. } => Some(sym),
+            _ => None,
+        }
+    }
+}
+
+/// A pattern in a term argument. Adds "evaluated expression" to kinds
+/// of patterns in addition to all options in `Pattern`.
+#[derive(Clone, PartialEq, Eq, Debug)]
+pub enum TermArgPattern {
+    /// A regular pattern that must match the existing value in the term's argument.
+    Pattern(Pattern),
+    /// An expression that is evaluated during the match phase and can
+    /// be given into an extractor. This is essentially a limited form
+    /// of unification or bidirectional argument flow (a la Prolog):
+    /// we can pass an arg *into* an extractor rather than getting the
+    /// arg *out of* it.
+    Expr(Expr),
+}
+
 /// An expression: the right-hand side of a rule.
 ///
 /// Note that this *almost* looks like a core Lisp or lambda calculus,
@@ -124,8 +162,15 @@ pub enum Extern {
         func: Ident,
         /// The position of this decl.
         pos: Pos,
-        /// Whether this extractor is infallible (always matches).
-        infallible: bool,
+        /// Poliarity of args: whether values are inputs or outputs to
+        /// the external extractor function. This is a sort of
+        /// statically-defined approximation to Prolog-style
+        /// unification; we allow for the same flexible directionality
+        /// but fix it at DSL-definition time. By default, every arg
+        /// is an *output* from the extractor (and the 'retval", or
+        /// more precisely the term value that we are extracting, is
+        /// an "input").
+        arg_polarity: Option<Vec<ArgPolarity>>,
     },
     /// An external constructor: `(constructor Term rustfunc)` form.
     Constructor {
@@ -137,3 +182,13 @@ pub enum Extern {
         pos: Pos,
     },
 }
+
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+pub enum ArgPolarity {
+    /// An arg that must be given an Expr in the pattern and passes
+    /// data *to* the extractor op.
+    Input,
+    /// An arg that must be given a regular pattern (not Expr) and
+    /// receives data *from* the extractor op.
+    Output,
+}