2be12a5167
* egraph-based midend: draw the rest of the owl. * Rename `egg` submodule of cranelift-codegen to `egraph`. * Apply some feedback from @jsharp during code walkthrough. * Remove recursion from find_best_node by doing a single pass. Rather than recursively computing the lowest-cost node for a given eclass and memoizing the answer at each eclass node, we can do a single forward pass; because every eclass node refers only to earlier nodes, this is sufficient. The behavior may slightly differ from the earlier behavior because we cannot short-circuit costs to zero once a node is elaborated; but in practice this should not matter. * Make elaboration non-recursive. Use an explicit stack instead (with `ElabStackEntry` entries, alongside a result stack). * Make elaboration traversal of the domtree non-recursive/stack-safe. * Work analysis logic in Cranelift-side egraph glue into a general analysis framework in cranelift-egraph. * Apply static recursion limit to rule application. * Fix aarch64 wrt dynamic-vector support -- broken rebase. * Topo-sort cranelift-egraph before cranelift-codegen in publish script, like the comment instructs me to! * Fix multi-result call testcase. * Include `cranelift-egraph` in `PUBLISHED_CRATES`. * Fix atomic_rmw: not really a load. * Remove now-unnecessary PartialOrd/Ord derivations. * Address some code-review comments. * Review feedback. * Review feedback. * No overlap in mid-end rules, because we are defining a multi-constructor. * rustfmt * Review feedback. * Review feedback. * Review feedback. * Review feedback. * Remove redundant `mut`. * Add comment noting what rules can do. * Review feedback. * Clarify comment wording. * Update `has_memory_fence_semantics`. * Apply @jameysharp's improved loop-level computation. Co-authored-by: Jamey Sharp <jamey@minilop.net> * Fix suggestion commit. * Fix off-by-one in new loop-nest analysis. * Review feedback. * Review feedback. * Review feedback. * Use `Default`, not `std::default::Default`, as per @fitzgen Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com> * Apply @fitzgen's comment elaboration to a doc-comment. Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com> * Add stat for hitting the rewrite-depth limit. * Some code motion in split prelude to make the diff a little clearer wrt `main`. * Take @jameysharp's suggested `try_into()` usage for blockparam indices. Co-authored-by: Jamey Sharp <jamey@minilop.net> * Take @jameysharp's suggestion to avoid double-match on load op. Co-authored-by: Jamey Sharp <jamey@minilop.net> * Fix suggestion (add import). * Review feedback. * Fix stack_load handling. * Remove redundant can_store case. * Take @jameysharp's suggested improvement to FuncEGraph::build() logic Co-authored-by: Jamey Sharp <jamey@minilop.net> * Tweaks to FuncEGraph::build() on top of suggestion. * Take @jameysharp's suggested clarified condition Co-authored-by: Jamey Sharp <jamey@minilop.net> * Clean up after suggestion (unused variable). * Fix loop analysis. * loop level asserts * Revert constant-space loop analysis -- edge cases were incorrect, so let's go with the simple thing for now. * Take @jameysharp's suggestion re: result_tys Co-authored-by: Jamey Sharp <jamey@minilop.net> * Fix up after suggestion * Take @jameysharp's suggestion to use fold rather than reduce Co-authored-by: Jamey Sharp <jamey@minilop.net> * Fixup after suggestion * Take @jameysharp's suggestion to remove elaborate_eclass_use's return value. * Clarifying comment in terminator insts. Co-authored-by: Jamey Sharp <jamey@minilop.net> Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
208 lines
6.8 KiB
Common Lisp
208 lines
6.8 KiB
Common Lisp
;; Algebraic optimizations.
|
|
|
|
;; Rules here are allowed to rewrite pure expressions arbitrarily,
|
|
;; using the same inputs as the original, or fewer. In other words, we
|
|
;; cannot pull a new eclass id out of thin air and refer to it, other
|
|
;; than a piece of the input or a new node that we construct; but we
|
|
;; can freely rewrite e.g. `x+y-y` to `x`.
|
|
|
|
;; uextend/sextend of a constant.
|
|
(rule (simplify (uextend $I64 (iconst $I32 imm)))
|
|
(iconst $I64 imm))
|
|
(rule (simplify (sextend $I64 (iconst $I32 (u64_from_imm64 imm))))
|
|
(iconst $I64 (imm64 (u64_sextend_u32 imm))))
|
|
|
|
;; x+0 == 0+x == x.
|
|
(rule (simplify (iadd ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(subsume x))
|
|
(rule (simplify (iadd ty
|
|
(iconst ty (u64_from_imm64 0))
|
|
x))
|
|
(subsume x))
|
|
;; x-0 == x.
|
|
(rule (simplify (isub ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(subsume x))
|
|
;; 0-x == (ineg x).
|
|
(rule (simplify (isub ty
|
|
(iconst ty (u64_from_imm64 0))
|
|
x))
|
|
(ineg ty x))
|
|
|
|
;; x*1 == 1*x == x.
|
|
(rule (simplify (imul ty
|
|
x
|
|
(iconst ty (u64_from_imm64 1))))
|
|
(subsume x))
|
|
(rule (simplify (imul ty
|
|
(iconst ty (u64_from_imm64 1))
|
|
x))
|
|
(subsume x))
|
|
|
|
;; x*0 == 0*x == x.
|
|
(rule (simplify (imul ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(iconst ty (imm64 0)))
|
|
(rule (simplify (imul ty
|
|
(iconst ty (u64_from_imm64 0))
|
|
x))
|
|
(iconst ty (imm64 0)))
|
|
|
|
;; x/1 == x.
|
|
(rule (simplify (sdiv ty
|
|
x
|
|
(iconst ty (u64_from_imm64 1))))
|
|
(subsume x))
|
|
(rule (simplify (udiv ty
|
|
x
|
|
(iconst ty (u64_from_imm64 1))))
|
|
(subsume x))
|
|
|
|
;; x>>0 == x<<0 == x rotr 0 == x rotl 0 == x.
|
|
(rule (simplify (ishl ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(subsume x))
|
|
(rule (simplify (ushr ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(subsume x))
|
|
(rule (simplify (sshr ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(subsume x))
|
|
(rule (simplify (rotr ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(subsume x))
|
|
(rule (simplify (rotl ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(subsume x))
|
|
|
|
;; x | 0 == 0 | x == x | x == x.
|
|
(rule (simplify (bor ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(subsume x))
|
|
(rule (simplify (bor ty
|
|
(iconst ty (u64_from_imm64 0))
|
|
x))
|
|
(subsume x))
|
|
(rule (simplify (bor ty x x))
|
|
(subsume x))
|
|
|
|
;; x ^ 0 == 0 ^ x == x.
|
|
(rule (simplify (bxor ty
|
|
x
|
|
(iconst ty (u64_from_imm64 0))))
|
|
(subsume x))
|
|
(rule (simplify (bxor ty
|
|
(iconst ty (u64_from_imm64 0))
|
|
x))
|
|
(subsume x))
|
|
|
|
;; x ^ x == 0.
|
|
(rule (simplify (bxor ty x x))
|
|
(subsume (iconst ty (imm64 0))))
|
|
|
|
;; x ^ not(x) == not(x) ^ x == -1.
|
|
(rule (simplify (bxor $I32 x (bnot $I32 x))) (subsume (iconst $I32 (imm64 0xffff_ffff))))
|
|
(rule (simplify (bxor $I32 (bnot $I32 x) x)) (subsume (iconst $I32 (imm64 0xffff_ffff))))
|
|
(rule (simplify (bxor $I64 x (bnot $I64 x))) (subsume (iconst $I64 (imm64 0xffff_ffff_ffff_ffff))))
|
|
(rule (simplify (bxor $I64 (bnot $I64 x) x)) (subsume (iconst $I64 (imm64 0xffff_ffff_ffff_ffff))))
|
|
|
|
;; x & -1 == -1 & x == x & x == x.
|
|
(rule (simplify (band ty x x)) x)
|
|
(rule (simplify (band $I32 x (iconst $I32 (u64_from_imm64 0xffff_ffff)))) (subsume x))
|
|
(rule (simplify (band $I32 (iconst $I32 (u64_from_imm64 0xffff_ffff)) x)) (subsume x))
|
|
(rule (simplify (band $I64 x (iconst $I64 (u64_from_imm64 0xffff_ffff_ffff_ffff)))) (subsume x))
|
|
(rule (simplify (band $I64 (iconst $I64 (u64_from_imm64 0xffff_ffff_ffff_ffff)) x)) (subsume x))
|
|
|
|
;; x & 0 == 0 & x == 0.
|
|
(rule (simplify (band ty x (iconst ty (u64_from_imm64 0)))) (iconst ty (imm64 0)))
|
|
(rule (simplify (band ty (iconst ty (u64_from_imm64 0)) x)) (iconst ty (imm64 0)))
|
|
|
|
;; not(not(x)) == x.
|
|
(rule (simplify (bnot ty (bnot ty x))) (subsume x))
|
|
|
|
;; DeMorgan's rule (two versions):
|
|
;; bnot(bor(x, y)) == band(bnot(x), bnot(y))
|
|
(rule (simplify (bnot ty (bor ty x y)))
|
|
(band ty (bnot ty x) (bnot ty y)))
|
|
;; bnot(band(x, y)) == bor(bnot(x), bnot(y))
|
|
(rule (simplify (bnot ty (band t x y)))
|
|
(bor ty (bnot ty x) (bnot ty y)))
|
|
|
|
;; x*2 == 2*x == x+x.
|
|
(rule (simplify (imul ty x (iconst _ (simm32 2))))
|
|
(iadd ty x x))
|
|
(rule (simplify (imul ty (iconst _ (simm32 2)) x))
|
|
(iadd ty x x))
|
|
|
|
;; x<<32>>32: uextend/sextend 32->64.
|
|
(rule (simplify (ushr $I64 (ishl $I64 (uextend $I64 x @ (eclass_type $I32)) (iconst _ (simm32 32))) (iconst _ (simm32 32))))
|
|
(uextend $I64 x))
|
|
|
|
(rule (simplify (sshr $I64 (ishl $I64 (uextend $I64 x @ (eclass_type $I32)) (iconst _ (simm32 32))) (iconst _ (simm32 32))))
|
|
(sextend $I64 x))
|
|
|
|
;; TODO: strength reduction: mul/div to shifts
|
|
;; TODO: div/rem by constants -> magic multiplications
|
|
|
|
;; Reassociate when it benefits LICM.
|
|
(rule (simplify (iadd ty (iadd ty x y) z))
|
|
(if-let (at_loop_level lx _) x)
|
|
(if-let (at_loop_level ly _) y)
|
|
(if-let (at_loop_level lz _) z)
|
|
(if (u8_lt lx ly))
|
|
(if (u8_lt lz ly))
|
|
(iadd ty (iadd ty x z) y))
|
|
(rule (simplify (iadd ty (iadd ty x y) z))
|
|
(if-let (at_loop_level lx _) x)
|
|
(if-let (at_loop_level ly _) y)
|
|
(if-let (at_loop_level lz _) z)
|
|
(if (u8_lt ly lx))
|
|
(if (u8_lt lz lx))
|
|
(iadd ty (iadd ty y z) x))
|
|
|
|
;; Select's selector input doesn't need bint; remove the redundant op.
|
|
(rule (simplify (select ty (bint _ b) x y))
|
|
(subsume (select ty b x y)))
|
|
|
|
;; Rematerialize ALU-op-with-imm and iconsts in each block where they're
|
|
;; used. This is neutral (add-with-imm) or positive (iconst) for
|
|
;; register pressure, and these ops are very cheap.
|
|
(rule (simplify x @ (iadd _ (iconst _ _) _))
|
|
(remat x))
|
|
(rule (simplify x @ (iadd _ _ (iconst _ _)))
|
|
(remat x))
|
|
(rule (simplify x @ (isub _ (iconst _ _) _))
|
|
(remat x))
|
|
(rule (simplify x @ (isub _ _ (iconst _ _)))
|
|
(remat x))
|
|
(rule (simplify x @ (band _ (iconst _ _) _))
|
|
(remat x))
|
|
(rule (simplify x @ (band _ _ (iconst _ _)))
|
|
(remat x))
|
|
(rule (simplify x @ (bor _ (iconst _ _) _))
|
|
(remat x))
|
|
(rule (simplify x @ (bor _ _ (iconst _ _)))
|
|
(remat x))
|
|
(rule (simplify x @ (bxor _ (iconst _ _) _))
|
|
(remat x))
|
|
(rule (simplify x @ (bxor _ _ (iconst _ _)))
|
|
(remat x))
|
|
(rule (simplify x @ (bnot _ _))
|
|
(remat x))
|
|
(rule (simplify x @ (iconst _ _))
|
|
(remat x))
|
|
(rule (simplify x @ (f32const _ _))
|
|
(remat x))
|
|
(rule (simplify x @ (f64const _ _))
|
|
(remat x))
|