Chris Fallin
e1f67e860f
Pinned VRegs for use with regalloc.rs shim to support RealRegs.
2021-05-18 22:40:43 -07:00
Chris Fallin
04c8e46787
Only do annotations in debug builds
2021-05-18 18:52:34 -07:00
Chris Fallin
c3513b94b0
Bugfix: don't do a split-at-intermediate-defs split if the first such point is the start of the bundle.
2021-05-18 15:16:19 -07:00
Chris Fallin
4389f16156
debugging log message for liveins
2021-05-18 12:14:59 -07:00
Chris Fallin
8e0d0f1de0
fuzzbug fix
2021-05-18 00:26:38 -07:00
Chris Fallin
328c9004e5
fuzzbug fixes
2021-05-17 23:59:13 -07:00
Chris Fallin
f0fbf3aa0c
Rework data structures: bundles have a SmallVec of ranges, and ranges a SmallVec of uses.
...
Appears to be a small speed improvement on the highly-artificial
fuzz-generator test inputs; Cranelift tests TBD.
2021-05-17 22:44:10 -07:00
Chris Fallin
5b55948feb
Check branch-args for conflicts with edge-move placement.
2021-05-13 17:25:11 -07:00
Chris Fallin
1f9258bea5
Detect undefined liveins.
2021-05-12 01:06:27 -07:00
Chris Fallin
37fa3ec763
Improve prog-move handling: no use/def records, just directly connect the LRs.
...
Also requires some metadata in edit output to properly hook up the
checker in regalloc.rs to track user-moves without seeing the original
insts with operands.
2021-05-11 23:59:12 -07:00
Chris Fallin
6066d02f6f
More annotations
2021-05-11 18:19:40 -07:00
Chris Fallin
b069ae099d
Use hot-code map to augment spill weights of each use
2021-05-11 17:59:10 -07:00
Chris Fallin
e1a37cf0e0
some more stats
2021-05-10 22:53:44 -07:00
Chris Fallin
f7551c68d1
Integrate prog-moves with LR-moves; this should in theory reduce move traffic somewhat
2021-05-10 22:47:57 -07:00
Chris Fallin
0dbf4a790f
Collect full conflict-bundle list, by not ending PhysReg probe on first conflict; this leads to better eviction decisions on bz2
2021-05-09 20:21:57 -07:00
Chris Fallin
b7fd53efc5
Fix checker: after moving edge-moves to prior to last branch of block (for simpler semantics for library user), we can no longer check blockparams; but this is fine because they do not exist in post-regalloc code.
2021-05-09 19:47:37 -07:00
Chris Fallin
4f26b1c78f
Properly handle prog-moves with fixed srcs or dests
2021-05-09 13:35:38 -07:00
Chris Fallin
5c5ea4cb9b
bugfix
2021-05-09 04:11:30 -07:00
Chris Fallin
34421fcc6b
fix to prog-move handling: happens in middle of inst; and insert uses to make later move-insertion happy with this
2021-05-09 03:51:10 -07:00
Chris Fallin
8d7530d3fa
Edge moves always before jumps, never after; semantics are too subtle otherwise (client needs to handle specially)
2021-05-09 02:20:38 -07:00
Chris Fallin
c380b0d979
assert fix: RegClass doesn't need to match for spillslots (can be reused across classes)
2021-05-09 01:51:00 -07:00
Chris Fallin
9fdc69edde
fuzzbug fix in range-summary iter
2021-05-09 01:48:16 -07:00
Chris Fallin
509c5dc2fd
Remove sanity-check logic in range summary construction -- zero-length
...
ranges make this somewhat fickle to verify, and fuzzing will catch any
issues.
2021-05-09 01:15:34 -07:00
Chris Fallin
095a883814
Fix crit-edge detection logic in CFGInfo analysis
2021-05-09 01:06:59 -07:00
Chris Fallin
b9e89885c4
Error checking: properly signal a crit-edge requirement failure (used for regalloc.rs fuzzer)
2021-05-08 21:48:58 -07:00
Chris Fallin
f1fc9a8f7e
Fix related to move handling
2021-05-08 19:04:16 -07:00
Chris Fallin
00c64f680a
Handle moves by joining LRs at inst boundary, not middle of move inst
2021-05-08 17:45:24 -07:00
Chris Fallin
3d0d760c70
Bugfix: process parallel moves separately for Int and Float classes
2021-05-08 16:16:30 -07:00
Chris Fallin
179ef0e534
Bugfix: Mod with dead def spans both Before and After positions
2021-05-08 15:47:38 -07:00
Chris Fallin
ed339ab4d8
Some minor opts: inlining, and smallvec reuse
2021-05-07 20:54:27 -07:00
Chris Fallin
e41b0101a8
Struct-of-array transform: pull LiveRangeHot out of LiveRange with just range and next-in-bundle link
2021-05-07 20:41:33 -07:00
Chris Fallin
4185eab441
More efficient live-range creation re: uses
2021-05-07 20:12:40 -07:00
Chris Fallin
a6e3128821
Support mod (modify) operands, for better efficiency with regalloc.rs/Cranelift shim.
2021-05-07 19:48:34 -07:00
Chris Fallin
d2cc4f1ac2
More efficient queue_bundles (saves 18% on clang.wasm)
2021-05-07 19:20:28 -07:00
Chris Fallin
040c3c838c
Some structure packing: Use now fits in three u32s.
2021-05-07 19:05:20 -07:00
Chris Fallin
bfe1c632c9
Some preallocation and removal of one u32 from LiveRange struct
2021-05-07 18:52:46 -07:00
Chris Fallin
a453501ebb
sort_unstable (quicksort) everywhere
2021-05-07 18:17:13 -07:00
Chris Fallin
df59b5ede4
Inline all the things (ProgPoint edition)
2021-05-07 17:55:04 -07:00
Chris Fallin
4f6346768e
Pinned-VReg mechanism.
2021-05-07 17:45:51 -07:00
Chris Fallin
3ddcf05fea
Optimizations: (i) range-summary array; (ii) early exit from btree probe loop (one conflict bundle is enough, empirically)
2021-05-07 17:03:44 -07:00
Chris Fallin
0f3454b4d7
Inlining on btree commitment map comparators for a 10% win
2021-05-07 01:51:40 -07:00
Chris Fallin
3713d6131e
Replace approximate liveness with true iterative liveness; turns out it is better to improve accuracy so that later stages of the allocator have less wasted work/interference
2021-05-07 01:22:12 -07:00
Chris Fallin
42582e0c6f
Some stats for loop effects on liveins: 487k loop set-unions (441 loops) in one func in bz2 -- fix TBD
2021-05-07 00:19:41 -07:00
Chris Fallin
2ba518517d
Fuzzbugfix: actually do need eager liveness computation; must uphold invariant that all earlier-in-postorder blocks have full livein sets.
2021-05-06 23:29:59 -07:00
Chris Fallin
2ff02b50a3
Some perf opts in liveness computation and set impl:
...
- Cache the most recent u64 chunk in the set to avoid some hashmap
lookups;
- Defer the live-set union'ing over the loop body until query time
(remember the set that would have been union'd in instead), and lazily
propagate the liveness bit at that query time, union-find style;
- Do n-1 rather than n union operations for n successors (first is a
clone instead);
- Don't union in liveness sets from blocks we haven't visited yet (the
loop-body/backedge handling handles these).
2021-05-06 22:46:16 -07:00
Chris Fallin
a148dccac3
Parameterize adaptive-map size in BitVec.
2021-05-06 22:14:46 -07:00
Chris Fallin
02b6516acd
Some memory-size/bitpacking optimizations
2021-05-06 20:47:17 -07:00
Chris Fallin
07a5a88972
BitVec perf: use adaptive hybrid chunked small-array + FxHashMap.
2021-05-06 20:03:44 -07:00
Chris Fallin
e2beb471c4
Handle moves specially with move-insertion logic rather than ordinary operand/inst handling
2021-05-06 18:49:23 -07:00
Chris Fallin
747c56c2c3
Some micro-optimizations in BitVec.
2021-05-06 16:19:38 -07:00
