Define two public iterator types in the flowgraph module, PredIter and
SuccIter, which are by-value iterators over an EBB's predecessors and
successors respectively.
Provide matching pred_iter() and succ_iter() methods for inspecting the
CFG. Remove the get_predecessors() method which returned a slice.
Update the uses of get_predecessors(), none of which depended on it
being a slice.
This abstraction makes it possible to change the internal representation
of the CFG.
Fixes#178.
When an instruction with a fixed output operand defines a globally live
SSA value, we need to check if the fixed register is available in the
`regs.global` set of registers that can be used across EBB boundaries.
If the fixed output register is not available in regs.global, set the
replace_global_defines flag so the output operands are rewritten as
local values.
Fixes#175.
The Intel division instructions have fixed input operands that are
clobbered by fixed output operands, so the value passed as an input will
be clobbered just like a tied operand.
The FixedTied operand constraint is used to indicate a fixed input
operand that has a corresponding output operand with the same fixed
register.
Teach the spiller to teach a FixedTied operand the same as a Tied
operand constraint and make sure that the input value is killed by the
instruction.
Rename the ArgumentType type to AbiParam since it describes the ABI
characteristics of a parameter or return value, not just the value type.
In Signature, rename members argument_types and return_types to "params"
and "returns". Again, they are not just types.
Fix a couple lingering references to "EBB arguments".
Add EBB parameter and EBB argument to the langref glossary to clarify
the distinction between formal EBB parameter values and arguments passed
to branches.
- Replace "ebb_arg" with "ebb_param" in function names that deal with
EBB parameters.
- Rename the ValueDef variants to Result and Param.
- A bunch of other small langref fixes.
No functional changes intended.
This makes it possible to define register banks that opt out of register
pressure tracking. This will be used to define banks for special-purpose
registers like the CPU flags.
The pressure tracker does not need to use resources for a top-level
register class in a non-tracked bank. The constant MAX_TOPRCS is renamed
to MAX_TRACKED_TOPRCS to indicate that there may be top-level register
classes with higher numbers, but they won't require pressure tracking.
We won't be tracking register pressure for CPU flags since only one
value is allowed to be live at a time.
Track allocatable registers both locally and globally: Add a second
AllocatableSet which tracks registers allocated to global values without
accounting for register diversions. Since diversions are only local to
an EBB, global values must be assigned un-diverted locations that don't
interfere.
Handle the third "global" interference domain in the constraint solver in
addition to the existing "input" and "output" domains.
Extend the solver error code to indicate when a global define just can't
be allocated because there are not enough available global registers.
Resolve this problem by replacing the instruction's global defines with
local defines that are copied into their global destinations
afterwards.
During iterate_solution(), live-through values may be converted to
solver variables so they can be moved out of the way in order to satisfy
all constraints. Make sure that the instruction's operand constraints
are also considered for these new variables.
Add a program_complete_input_constraints() which turns all the
instruction's input operands into variables with the proper constraints.
That makes it safe for try_add_var() to re-add these values as variables
with looser generic constraints.
The solver's add_var() function is split into three functions: add_var
for use before inputs_done(), and add_killed_var/add_through_var for use
after.
When solver variables represent operands on the current instruction,
they need to be constrained as required by the instructions, but
variables that are simply moved out of the way should only be
constrained to their top-level register class. The live range affinity
is just a hint, not a requirement.
When try_add_var is looking for values that can be moved out of the way
in order to satisfy constraints for the current instruction, avoid
values that are live on a CFG edge originating at the current (branch)
instruction.
These values must be in their globally assigned location when entering
the branch destination EBB.
This is covered by the existing regalloc/iterate.cton test case which
fails with an upcoming commit.
- Create a new kind of stack slot: emergency_slot.
- Add a get_emergency_slot() method which finds a suitable emergency
slot given a list of slots already in use.
- Use emergency spill slots when schedule_moves needs them.
This is a verification pass that can be run after register allocation.
It verifies that value locations are consistent with constraints on
their uses, and that the register diversions are consistent.
Make it clear that register diversions are local to an EBB only. This
affects what branch relaxation is allowed to do.
The verify_locations() takes an optional Liveness parameter which is
used to check that no diverted values are live across CFG edges.
When "binemit" tests encode instructions, keep track of the current set
of register diversions, and use the diverted locations to check operand
constraints.
This matches how constraints are applied during a real binemit phase.
The register constraint solver schedules a set of move instructions to
execute before the instruction getting colored. In extreme cases, this
is not possible because there are no available registers to break cycles
in the register assignments that must be scheduled.
When that happens, we spill one register to an emergency slot so it
becomes available for implementing the assignment cycle. Then the
original register is restored.
The coloring pass can't yet understand the spill and fill move types.
This will be implemented next.
This makes it possible to materialize new RegClass references without
requiring a RegInfo reference to be passed around.
- Move the RegInfo::toprc() method to RegClassData.
- Rename RegClassData::intersect() to intersect_index() and provide a
new intersect() which returns a register class.
- Remove some &RegInfo parameters that are no longer needed.
Fixes#165.
The constraint solver's schedule_move() function sometimes need to use
an extra available register when the moves to be scheduled contains
cycles.
The pending moves have associated register classes that come from the
constraint programming. Since the moves have hard-coded to and from
registers, these register classes are only meant to indicate the
register sizes. In particular, we can use the whole top-level register
class when scavenging for a spare register to break a cycle.
When we detect interference between the values that have already been
merged into the candidate virtual register and an EBB argument, we first
try to resolve the conflict by splitting. We also check if the existing
interfering value is fundamentally incompatible with the branch
instruction so it needs to be removed from the virtual register,
restarting the merge operation.
However, this existing interfering value is not necessarily the only
interference, so the split is not guaranteed to resolve the conflict. If
it turns out that splitting didn't resolve the conflict, restart the
merge after removing this second conflicting value.
It can happen that the currently live registers are blocking a smaller
register class completely, so the only way of solving the allocation
problem is to turn some of the live-through registers into solver
variables.
When the quick_solve attempt fails, try to free up registers in the
critical register class by turning live-through values into solver
variables.
When the return value from a call has been spilled, the reload pass
needs to insert a spill instruction right after the call instruction
which returns its results in registers.
A cursor now also remembers a current source location which will be
assigned to all new instructions created with the cursor.
The old layout::Cursor can't support source locations because it doesn't
have a reference to the full ir::Function.
Add a settings::FlagsOrIsa struct which represents a flags reference and
optionally the ISA it belongs to. Use this for passing flags/isa
information to the verifier.
The verify_function() and verify_context() functions are now generic so
they accept either a &Flags or a &TargetISa argument.
Fix the return_at_end verifier tests which no longer require an ISA
specified. The signle "set return_at_end" flag setting now makes it to
the verifier even when no ISA is present to carry it.
* Make passes assert their dependencies consistently.
This avoids ambiguity about whose responsibility it is to run
to compute cfg, domtree, and loop_analysis data.
* Reset the `valid` flag in DominatorTree's `clear()`.
* Remove the redundant assert from DominatorTree::with_function.
* Remove the message strings from obvious asserts.
This avoids having them spill out into multiple lines.
* Refactor calls to `compute` on `Context` objects into helper functions.
This also moves the calls to it out of Context and into the passes that
actually need it, so that Context's functions don't have any logic of
their own.
Fixes#147.
The Solver::reassign_in() method would previously not record fixed
register assignments for values that are already in the correct
register. The register would simply be marked as unavailable for the
solver.
This did have the effect of tripping up the sanity checks in
Solver::add_var() when that method was called with such a "reassigned"
value. The function can be called for a value that already has a fixed
assignment, but the sanity checks want to make sure the variable
constraints are compatible with the existing fixed assignment. When no
such assignment could be found, the method panicked.
To fix this, make sure that even identity reassignments are recorded
in the assignments vector. Instead, filter the identity assignments out
before scheduling a move sequence for the assignments.
Also add some debug tracing to the regalloc solver.
The new PrimaryMap replaces the primary EntityMap and the PrimaryEntityData
marker trait which was causing some confusion. We now have a clear
division between the two types of maps:
- PrimaryMap is used to assign entity numbers to the primary data for an
entity.
- EntityMap is a secondary mapping adding additional info.
The split also means that the secondary EntityMap can now behave as if
all keys have a default value. This means that we can get rid of the
annoying ensure() and get_or_default() methods ther were used everywhere
instead of indexing. Just use normal indexing now; non-existent keys
will return the default value.
Use an EncCursor instead of a layout cursor to keep track of the
current position in the function. Since the EncCursor holds a reference
to the whole IR function insteadof just the layout, we can rework how IR
borrowing works.
The Context data structure that's live during the spilling pass now owns
an EncCursor which in turn holds references to the function and ISA.
This means that we no longer need to pass around references to parts of
the ir::Function. We can no longer borrow any part of the IR function
across a context method call, but that turns out to be not necessary.
Add a new cursor module and define an EncCursor data type in it. An
EncCursor is a cursor that inserts instructions with a valid encoding
for the ISA. This is useful for passes generating code after
legalization.
Implement a builder interface via the new InstInserterBase trait such
that the EncCursor builders support with_result().
Use EncCursor in coalescing.rs instead of the layout cursor as a proof
of concept.
