Ghost instructions don't generate code, but they can keep registers
alive. The coloring pass needs to process values killed by ghost
instructions so it knows when the registers are freed up.
Also track register pressure changes from ghost kills in the spiller.
When the spiller decides to spill a value, bring along all of the values
in its virtual register. This ensures that we won't have problems with
computing register pressure around EBB arguments. They will always be
register-to-register or stack-to-stack with related values using the
same stack slot.
This also means that the reloading pass won't have to deal with spilled
EBB arguments.
Coalescing means creating virtual registers and transforming the code
into conventional SSA form. This means that every value used as a branch
argument will belong to the same virtual register as the corresponding
EBB argument value.
Conventional SSA form makes it easy to avoid memory-memory copies when
spilling values, and the virtual registers can be used as hints when
picking registers too. This reduces the number of register moves needed
for EBB arguments.
Add a VirtRegs collection which tracks virtual registers.
A virtual register is a set of related SSA values whose live ranges
don't interfere. It is advantageous to use the same register or spill
slot for al the values in a virtual register. It reduces copies for EBB
arguments.
Ghost instructions don't have an encoding, and don't appear in the
output. The values they define do not need to be assigned to registers,
so they can be skipped.
The overlaps_def() method tests if a definition would conflict with the
live range.
The reaches_use() method tests if a live range is live at an
instruction.
The EntityRef trait is used by more than just the EntityMap now, so it
should live in its own module.
Also move the entity_impl! macro into the new module so it can be used
for defining new entity references anywhere.
* Replace a single-character string literal with a character literal.
* Use is_some() instead of comparing with Some(_).
* Add code-quotes around type names in comments.
* Use !...is_empty() instead of len() != 0.
* Tidy up redundant returns.
* Remove redundant .clone() calls.
* Remove unnecessary explicit lifetime parameters.
* Tidy up unnecessary '&'s.
* Add parens to make operator precedence explicit.
* Use debug_assert_eq instead of debug_assert with ==.
* Replace a &Vec argument with a &[...].
* Replace `a = a op b` with `a op= b`.
* Avoid unnecessary closures.
* Avoid .iter() and .iter_mut() for iterating over containers.
* Remove unneeded qualification.
As soon as a value is spilled, also assign it to a spill slot.
For now, create a new spill slot for each spilled value. In the future,
values will be sharing spill slots of they are phi-related.
An instruction may have fixed operand constraints that make it
impossibly to use a single register value to satisfy two at a time.
Detect when the same value is used for multiple fixed register operands
and insert copies during the spilling pass.
Even if an argument is already in the correct register, make sure that
we detect conflicts by registering the no-op move. This also means that
the ABI argument register won't be turned into a variable for the
solver.
Add a spilling pass which lowers register pressure by assigning SSA
values to the stack. Important missing features:
- Resolve conflicts where an instruction uses the same value more than
once in incompatible ways.
- Deal with EBB arguments.
Fix bugs in the reload pass exposed by the first test case:
- Create live ranges for temporary registers.
- Set encodings on created spill and fill instructions.
The register pressure tracker now has to separate register counts: base
and transient.
The transient counts are used to track spikes of register pressure, such
as dead defs or temporary registers needed to satisfy instruction
constraints.
The base counts represent long-lived variables.
The reload pass inserts spill and fill instructions as needed so
instructions that operate on registers will never see a value with stack
affinity.
This is a very basic implementation, and we can't write good test cases
until we have a spilling pass.
The create_dead() methods can create a live range for a new value, and
extend_local() can extend a live range within an EBB where it is already
live.
This is enough to update liveness for new values as long as they stay
local to their EBB.
The spilling and reload passes need to ensure that the set of live
ranges with register affinity can always be assigned registers. The
register pressure tracker can count how many registers are in use for
each top-level register class and give guidance on the type of
registers that need to be spilled when limits are exceeded.
Pressure tracking is extra complicated for the arm32 floating point
register bank because there are multiple top-level register classes (S,
D, Q) competing for the same register units.
A top-level register class is one that has no sub-classes. It is
possible to have multiple top-level register classes in the same
register bank. For example, ARM's FPR bank has both D and Q top-level
register classes.
Number register classes such that all top-level register classes appear
as a contiguous sequence starting from 0. This will be used by the
register allocator when counting used registers per top-level register
class.
After finding a register solution, it need to be executed as a sequence
of regmove instructions. This often requires a topological ordering of
the moves so they don't conflict.
When the solution contains cycles, try to grab an available scratch
register to implement the copies. Panic if that fails (later, we'll
implement emergency spilling in this case).
Make sure we handle odd aliasing in the arm32 floating point register
bank. Not everything is a simple cycle in that case, so make sure we
don't assume so.
The register constraint solver has two kinds of variables:
1. Live values that were already in a register, and
2. Values defined by the instruction.
Make a record of the original register holding the first kind of value.
It is not necessary to to a second pass over the live values to update
the set of available registers. The color_args() and color_entry_args()
functions can do that in a single pass.
The live value tracker expects them to be there.
We may eventually delete dead arguments from internal EBBs, but at least
the entry block needs to be able to handle dead function arguments.
Provide a drop_dead_args() function which deletes them instead.
We still need to assign a register to dead EBB arguments, so they can't
just be ignored.
The live value tracker goes through the trouble of looking up the live
range for each value it tracks. We can cache a few more interesting
properties from the live range in the LiveValue struct.
Most of the time, register coloring is almost trivial: just pick
available registers for the values defined by the current instruction.
However, some instructions have register operand constraints, and it may
be necessary to move live registers around to satisfy the constraints.
Sometimes the instruction's own operands can interfere with each other
in a way that you can't just pick a register assignment for each output
in order.
This is complicated enough that it is worthwhile to represent as a
constraint satisfaction problem in a separate solver module. The
representation is chosen to be very fast in the common case where the
constraints are trivial to solve.
The current implementation is still incomplete, but as functional as the
code it's replacing. Missing features:
- Handle tied operand constraints.
- Handle ABI constraints on calls and return instructions.
- Execute a constraint solution by emitting regmove instructions.
- Handling register diversions before leaving the EBB.
The arguments to the entry block arrive in registers determined by the
ABI. This information is stored in the signature.
Use a separate function for coloring entry block arguments using the
signature information. We can't handle stack arguments yet.
This value affinity doesn't mean "whatever", it means that the value
does not interact with any real instructions, where "real" means
instructions that have a legal encoding.
Update the liveness verifier to check this property:
- Encoded instructions can only use real values.
- Encoded instructions define real values.
- Ghost instructions define ghost values.
This means that we can verify the basics with verify_context before
moving on to verifying the liveness information.
Live ranges are now verified immediately after computing them and after
register allocation is complete.
The liveness verifier will check that the live ranges are consistent
with the function. It runs as part of the register allocation pipeline
when enable_verifier is set.
The initial implementation checks the live ranges, but not the
ISA-specific constraints and affinities.
Now we can access instruction results and arguments as well as EBB
arguments as slices.
Delete the Values iterator which was traversing the linked lists of
values. It is no longer needed.
This is the first step of the value list refactoring which will replace
linked lists of values with value lists.
- Keep a ValueList in the EbbData struct containing all the EBB
arguments.
- Change dfg.ebb_args() to return a slice instead of an iterator.
This leaves us in a temporary hybrid state where we maintain both a
linked list and a ValueList vector of the EBB arguments.
The tables returned by recipe_names() and recipe_constraints() are now
collected into an EncInfo struct that is available from
TargetIsa::encoding_info(). This is equivalent to the register bank
tables available fro TargetIsa::register_info().
This cleans of the TargetIsa interface and makes it easier to add
encoding-related information.
