Fixes#147.
The Solver::reassign_in() method would previously not record fixed
register assignments for values that are already in the correct
register. The register would simply be marked as unavailable for the
solver.
This did have the effect of tripping up the sanity checks in
Solver::add_var() when that method was called with such a "reassigned"
value. The function can be called for a value that already has a fixed
assignment, but the sanity checks want to make sure the variable
constraints are compatible with the existing fixed assignment. When no
such assignment could be found, the method panicked.
To fix this, make sure that even identity reassignments are recorded
in the assignments vector. Instead, filter the identity assignments out
before scheduling a move sequence for the assignments.
Also add some debug tracing to the regalloc solver.
The new PrimaryMap replaces the primary EntityMap and the PrimaryEntityData
marker trait which was causing some confusion. We now have a clear
division between the two types of maps:
- PrimaryMap is used to assign entity numbers to the primary data for an
entity.
- EntityMap is a secondary mapping adding additional info.
The split also means that the secondary EntityMap can now behave as if
all keys have a default value. This means that we can get rid of the
annoying ensure() and get_or_default() methods ther were used everywhere
instead of indexing. Just use normal indexing now; non-existent keys
will return the default value.
Use an EncCursor instead of a layout cursor to keep track of the
current position in the function. Since the EncCursor holds a reference
to the whole IR function insteadof just the layout, we can rework how IR
borrowing works.
The Context data structure that's live during the spilling pass now owns
an EncCursor which in turn holds references to the function and ISA.
This means that we no longer need to pass around references to parts of
the ir::Function. We can no longer borrow any part of the IR function
across a context method call, but that turns out to be not necessary.
Add a new cursor module and define an EncCursor data type in it. An
EncCursor is a cursor that inserts instructions with a valid encoding
for the ISA. This is useful for passes generating code after
legalization.
Implement a builder interface via the new InstInserterBase trait such
that the EncCursor builders support with_result().
Use EncCursor in coalescing.rs instead of the layout cursor as a proof
of concept.
The Cursor navigation methods all just depend on the cursor's position
and layout reference. Make a CursorBase trait that provides access to
this information with methods and implement the navigation methods on
top of that.
This makes it possible to have multiple types implement the cursor
interface.
When making an outgoing call, some arguments may have to be passed on
the stack. Allocate OutgoingArg stack slots for these arguments and
write them immediately before the outgoing call instruction.
Do the same for incoming function arguments on the stack, but use
IncomingArg stack slots instead. This was previously done in the
spiller, but we move it to the legalizer so it is done at the same time
as outgoing stack arguments.
These stack slot assignments are done in the legalizer before live
range analysis because the outgoing arguments usually are in different
SSSA values with their own short live ranges.
Replace the isa::Legalize enumeration with a function pointer. This
allows an ISA to define its own specific legalization actions instead of
relying on the default two.
Generate a LEGALIZE_ACTIONS table for each ISA which contains
legalization function pointers indexed by the legalization codes that
are already in the encoding tables. Include this table in
isa/*/enc_tables.rs.
Give the `Encodings` iterator a reference to the action table and change
its `legalize()` method to return a function pointer instead of an
ISA-specific code.
The Result<> returned from TargetIsa::encode() no longer implements
Debug, so eliminate uses of unwrap and expect on that type.
The following constraints may need to be resolved during spilling
because the resolution increases register pressure:
- A tied operand whose value is live through the instruction.
- A fixed register constraint for a value used more than once.
- A register use of a spilled value needs to account for the reload
register.
It is possible to pass a register value as an argument to an EBB that
expects a "None" affinity. In that case, the destination EBB value
should not be colored.
We'll need to pick a spill candidate from a set and allow for the search
to fail to find anything.
This also allows slightly better panic messages when we run out of
registers.
A priory, an EBB argument value only gets an affinity if it is used
directly by a non-ghost instruction. A use by a branch passing arguments
to an EBB doesn't count.
When an EBB argument value does have an affinity, the values passed by
all the predecessors must also have affinities. This can cause EBB
argument values to get affinities recursively.
- Add a second pass to the liveness computation for propagating EBB
argument affinities, possibly recursively.
- Verify EBB argument affinities correctly: A value passed to a branch
must have an affinity only if the corresponding EBB argument value in
the destination has an affinity.
When an EBB argument value is used only as a return value, it still
needs to be given a register affinity. Otherwise it would appear as a
ghost value with no affinity.
Do the same to call arguments.
A function parameter in an incoming_arg stack slot should not be
coalesced into any virtual registers. We don't want to force the whole
virtual register to spill to the incoming_arg slot.
Function arguments that don't fit in registers are passed on the stack.
Create "incoming_arg" stack slots representing the stack arguments, and
assign them to the value arguments during spilling.
When coloring registers for a branch instruction, also make sure that
the values passed as EBB arguments are in the registers expected by the
EBB.
The first time a branch to an EBB is processed, assign the EBB arguments
to the registers where the branch arguments already reside so no
regmoves are needed.
Ghost instructions don't generate code, but they can keep registers
alive. The coloring pass needs to process values killed by ghost
instructions so it knows when the registers are freed up.
Also track register pressure changes from ghost kills in the spiller.
When the spiller decides to spill a value, bring along all of the values
in its virtual register. This ensures that we won't have problems with
computing register pressure around EBB arguments. They will always be
register-to-register or stack-to-stack with related values using the
same stack slot.
This also means that the reloading pass won't have to deal with spilled
EBB arguments.
Coalescing means creating virtual registers and transforming the code
into conventional SSA form. This means that every value used as a branch
argument will belong to the same virtual register as the corresponding
EBB argument value.
Conventional SSA form makes it easy to avoid memory-memory copies when
spilling values, and the virtual registers can be used as hints when
picking registers too. This reduces the number of register moves needed
for EBB arguments.
Add a VirtRegs collection which tracks virtual registers.
A virtual register is a set of related SSA values whose live ranges
don't interfere. It is advantageous to use the same register or spill
slot for al the values in a virtual register. It reduces copies for EBB
arguments.
Ghost instructions don't have an encoding, and don't appear in the
output. The values they define do not need to be assigned to registers,
so they can be skipped.
The overlaps_def() method tests if a definition would conflict with the
live range.
The reaches_use() method tests if a live range is live at an
instruction.
The EntityRef trait is used by more than just the EntityMap now, so it
should live in its own module.
Also move the entity_impl! macro into the new module so it can be used
for defining new entity references anywhere.
* Replace a single-character string literal with a character literal.
* Use is_some() instead of comparing with Some(_).
* Add code-quotes around type names in comments.
* Use !...is_empty() instead of len() != 0.
* Tidy up redundant returns.
* Remove redundant .clone() calls.
* Remove unnecessary explicit lifetime parameters.
* Tidy up unnecessary '&'s.
* Add parens to make operator precedence explicit.
* Use debug_assert_eq instead of debug_assert with ==.
* Replace a &Vec argument with a &[...].
* Replace `a = a op b` with `a op= b`.
* Avoid unnecessary closures.
* Avoid .iter() and .iter_mut() for iterating over containers.
* Remove unneeded qualification.
As soon as a value is spilled, also assign it to a spill slot.
For now, create a new spill slot for each spilled value. In the future,
values will be sharing spill slots of they are phi-related.
An instruction may have fixed operand constraints that make it
impossibly to use a single register value to satisfy two at a time.
Detect when the same value is used for multiple fixed register operands
and insert copies during the spilling pass.
Even if an argument is already in the correct register, make sure that
we detect conflicts by registering the no-op move. This also means that
the ABI argument register won't be turned into a variable for the
solver.
Add a spilling pass which lowers register pressure by assigning SSA
values to the stack. Important missing features:
- Resolve conflicts where an instruction uses the same value more than
once in incompatible ways.
- Deal with EBB arguments.
Fix bugs in the reload pass exposed by the first test case:
- Create live ranges for temporary registers.
- Set encodings on created spill and fill instructions.
The register pressure tracker now has to separate register counts: base
and transient.
The transient counts are used to track spikes of register pressure, such
as dead defs or temporary registers needed to satisfy instruction
constraints.
The base counts represent long-lived variables.
The reload pass inserts spill and fill instructions as needed so
instructions that operate on registers will never see a value with stack
affinity.
This is a very basic implementation, and we can't write good test cases
until we have a spilling pass.
