T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Use a constraint solver for register coloring.

Browse Source 
Most of the time, register coloring is almost trivial: just pick
available registers for the values defined by the current instruction.
However, some instructions have register operand constraints, and it may
be necessary to move live registers around to satisfy the constraints.
Sometimes the instruction's own operands can interfere with each other
in a way that you can't just pick a register assignment for each output
in order.

This is complicated enough that it is worthwhile to represent as a
constraint satisfaction problem in a separate solver module. The
representation is chosen to be very fast in the common case where the
constraints are trivial to solve.

The current implementation is still incomplete, but as functional as the
code it's replacing. Missing features:

- Handle tied operand constraints.
- Handle ABI constraints on calls and return instructions.
- Execute a constraint solution by emitting regmove instructions.
- Handling register diversions before leaving the EBB.
This commit is contained in:
Jakob Stoklund Olesen
2017-04-28 15:41:28 -07:00
parent 2873019779
commit 6b4c28d554
4 changed files with 763 additions and 105 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
lib/cretonne/src/isa/encoding.rs
View File

@@ -104,7 +104,7 @@ pub struct EncInfo {
impl EncInfo {
/// Get the value operand constraints for `enc` if it is a legal encoding.
pub fn operand_constraints(&self, enc: Encoding) -> Option<&RecipeConstraints> {
pub fn operand_constraints(&self, enc: Encoding) -> Option<&'static RecipeConstraints> {
self.constraints.get(enc.recipe())
}

329
lib/cretonne/src/regalloc/coloring.rs
View File

@@ -15,7 +15,11 @@
//! values used by the tied operands must be killed by the instruction. This can be achieved by
//! inserting a `copy` to a new value immediately before the two-address instruction.
//!
//! 3. The register pressure must be lowered sufficiently by inserting spill code. Register
//! 3. If a value is bound to more than one operand on the same instruction, the operand
//! constraints must be compatible. This can also be achieved by inserting copies so the
//! incompatible operands get different values.
//!
//! 4. The register pressure must be lowered sufficiently by inserting spill code. Register
//! operands are allowed to read spilled values, but each such instance must be counted as using
//! a register.
//!
@@ -26,30 +30,28 @@
//! a topological order relative to the dominance relation, we can assign colors to the values
//! defined by the instruction and only consider the colors of other values that are live at the
//! instruction.
//!
//! The topological order of instructions inside an EBB is simply the layout order, starting from
//! the EBB header. A topological order of the EBBs can only visit an EBB once its immediate
//! dominator has been visited.
//!
//! There are many valid topological orders of the EBBs, and the specific order can affect which
//! coloring hints are satisfied and which are broken.
//!
use entity_map::EntityMap;
use dominator_tree::DominatorTree;
use ir::{Ebb, Inst, Value, Function, Cursor, ValueLoc, DataFlowGraph, Signature, ArgumentLoc};
use isa::{TargetIsa, RegInfo, Encoding, EncInfo, ConstraintKind};
use isa::{TargetIsa, Encoding, EncInfo, OperandConstraint, ConstraintKind};
use isa::{RegUnit, RegClass, RegInfo, regs_overlap};
use regalloc::affinity::Affinity;
use regalloc::allocatable_set::AllocatableSet;
use regalloc::live_value_tracker::{LiveValue, LiveValueTracker};
use regalloc::liveness::Liveness;
use regalloc::solver::Solver;
use regalloc::RegDiversions;
use topo_order::TopoOrder;
/// Data structures for the coloring pass.
///
/// These are scratch space data structures that can be reused between invocations.
pub struct Coloring {}
pub struct Coloring {
divert: RegDiversions,
solver: Solver,
}
/// Bundle of references that the coloring algorithm needs.
///
@@ -69,6 +71,13 @@ struct Context<'a> {
domtree: &'a DominatorTree,
liveness: &'a mut Liveness,
// References to working set data structures.
// If we need to borrow out of a data structure across a method call, it must be passed as a
// function argument instead, see the `LiveValueTracker` arguments.
topo: &'a mut TopoOrder,
divert: &'a mut RegDiversions,
solver: &'a mut Solver,
// Pristine set of registers that the allocator can use.
// This set remains immutable, we make clones.
usable_regs: AllocatableSet,
@@ -77,7 +86,10 @@ struct Context<'a> {
impl Coloring {
/// Allocate scratch space data structures for the coloring pass.
pub fn new() -> Coloring {
Coloring {}
Coloring {
divert: RegDiversions::new(),
solver: Solver::new(),
}
}
/// Run the coloring algorithm over `func`.
@@ -93,26 +105,31 @@ impl Coloring {
encinfo: isa.encoding_info(),
domtree,
liveness,
topo,
divert: &mut self.divert,
solver: &mut self.solver,
usable_regs: isa.allocatable_registers(func),
};
ctx.run(func, topo, tracker)
ctx.run(func, tracker)
}
}
impl<'a> Context<'a> {
/// Run the coloring algorithm.
fn run(&mut self, func: &mut Function, topo: &mut TopoOrder, tracker: &mut LiveValueTracker) {
// Just visit blocks in layout order, letting `topo` enforce a topological ordering.
fn run(&mut self, func: &mut Function, tracker: &mut LiveValueTracker) {
// Just visit blocks in layout order, letting `self.topo` enforce a topological ordering.
// TODO: Once we have a loop tree, we could visit hot blocks first.
topo.reset(func.layout.ebbs());
while let Some(ebb) = topo.next(&func.layout, self.domtree) {
self.topo.reset(func.layout.ebbs());
while let Some(ebb) = self.topo.next(&func.layout, self.domtree) {
self.visit_ebb(ebb, func, tracker);
}
}
/// Visit `ebb`, assuming that the immediate dominator has already been visited.
fn visit_ebb(&mut self, ebb: Ebb, func: &mut Function, tracker: &mut LiveValueTracker) {
dbg!("Coloring {}:", ebb);
let mut regs = self.visit_ebb_header(ebb, func, tracker);
self.divert.clear();
// Now go through the instructions in `ebb` and color the values they define.
let mut pos = Cursor::new(&mut func.layout);
@@ -168,10 +185,15 @@ impl<'a> Context<'a> {
.get(value)
.expect("No live range for live-in")
.affinity;
if let Affinity::Reg(rc_index) = affinity {
let regclass = self.reginfo.rc(rc_index);
match func.locations[value] {
ValueLoc::Reg(regunit) => regs.take(regclass, regunit),
if let Affinity::Reg(rci) = affinity {
let rc = self.reginfo.rc(rci);
let loc = func.locations[value];
dbg!("Live-in: {}:{} in {}",
lv.value,
rc,
loc.display(&self.reginfo));
match loc {
ValueLoc::Reg(reg) => regs.take(rc, reg),
ValueLoc::Unassigned => panic!("Live-in {} wasn't assigned", value),
ValueLoc::Stack(ss) => {
panic!("Live-in {} is in {}, should be register", value, ss)
@@ -200,11 +222,11 @@ impl<'a> Context<'a> {
for (lv, abi) in args.iter().zip(&sig.argument_types) {
match lv.affinity {
Affinity::Reg(rc_index) => {
let regclass = self.reginfo.rc(rc_index);
if let ArgumentLoc::Reg(regunit) = abi.location {
regs.take(regclass, regunit);
*locations.ensure(lv.value) = ValueLoc::Reg(regunit);
Affinity::Reg(rci) => {
let rc = self.reginfo.rc(rci);
if let ArgumentLoc::Reg(reg) = abi.location {
regs.take(rc, reg);
*locations.ensure(lv.value) = ValueLoc::Reg(reg);
} else {
// This should have been fixed by the reload pass.
panic!("Entry arg {} has {} affinity, but ABI {}",
@@ -247,14 +269,14 @@ impl<'a> Context<'a> {
-> AllocatableSet {
for lv in args {
// Only look at the register arguments.
if let Affinity::Reg(rc_index) = lv.affinity {
let regclass = self.reginfo.rc(rc_index);
if let Affinity::Reg(rci) = lv.affinity {
let rc = self.reginfo.rc(rci);
// TODO: Fall back to a top-level super-class. Sub-classes are only hints.
let regunit = regs.iter(regclass)
let reg = regs.iter(rc)
.next()
.expect("Out of registers for arguments");
regs.take(regclass, regunit);
*locations.ensure(lv.value) = ValueLoc::Reg(regunit);
regs.take(rc, reg);
*locations.ensure(lv.value) = ValueLoc::Reg(reg);
}
}
@@ -266,7 +288,7 @@ impl<'a> Context<'a> {
///
/// Update `regs` to reflect the allocated registers after `inst`, including removing any dead
/// or killed values from the set.
fn visit_inst(&self,
fn visit_inst(&mut self,
inst: Inst,
encoding: Encoding,
_pos: &mut Cursor,
@@ -274,95 +296,194 @@ impl<'a> Context<'a> {
tracker: &mut LiveValueTracker,
regs: &mut AllocatableSet,
locations: &mut EntityMap<Value, ValueLoc>) {
// First update the live value tracker with this instruction.
// Get lists of values that are killed and defined by `inst`.
let (_throughs, kills, defs) = tracker.process_inst(inst, dfg, self.liveness);
dbg!("Coloring [{}] {}",
self.encinfo.display(encoding),
dfg.display_inst(inst));
// Get the operand constraints for `inst` that we are trying to satisfy.
let constraints = self.encinfo
.operand_constraints(encoding)
.expect("Missing instruction encoding")
.clone();
.expect("Missing instruction encoding");
// Program the solver with register constraints for the input side.
self.solver.reset(regs);
self.program_input_constraints(inst, constraints.ins, dfg, locations);
if self.solver.has_fixed_input_conflicts() {
self.divert_fixed_input_conflicts(tracker.live(), locations);
}
self.solver.inputs_done();
// Update the live value tracker with this instruction.
let (throughs, kills, defs) = tracker.process_inst(inst, dfg, self.liveness);
// Get rid of the killed values.
for lv in kills {
if let Affinity::Reg(rc_index) = lv.affinity {
let regclass = self.reginfo.rc(rc_index);
if let ValueLoc::Reg(regunit) = locations[lv.value] {
regs.free(regclass, regunit);
}
if let Affinity::Reg(rci) = lv.affinity {
self.solver
.add_kill(lv.value,
self.reginfo.rc(rci),
self.divert.reg(lv.value, locations));
}
}
// Process the defined values with fixed constraints.
// TODO: Handle constraints on call return values.
assert_eq!(defs.len(),
constraints.outs.len(),
"Can't handle variable results");
for (lv, opcst) in defs.iter().zip(constraints.outs) {
match lv.affinity {
// This value should go in a register.
Affinity::Reg(rc_index) => {
// The preferred register class is not a requirement.
let pref_rc = self.reginfo.rc(rc_index);
match opcst.kind {
ConstraintKind::Reg => {
// This is a standard register constraint. The preferred register class
// should have been computed as a subclass of the hard constraint of
// the def.
assert!(opcst.regclass.has_subclass(rc_index),
"{} preference {} is not compatible with the definition \
constraint {}",
lv.value,
pref_rc.name,
opcst.regclass.name);
// Try to grab a register from the preferred class, but fall back to
// the actual constraint if we have to.
let regunit = regs.iter(pref_rc)
.next()
.or_else(|| regs.iter(opcst.regclass).next())
.expect("Ran out of registers");
regs.take(opcst.regclass, regunit);
*locations.ensure(lv.value) = ValueLoc::Reg(regunit);
}
ConstraintKind::Tied(arg_index) => {
// This def must use the same register as a fixed instruction argument.
let arg = dfg.inst_args(inst)[arg_index as usize];
let loc = locations[arg];
*locations.ensure(lv.value) = loc;
// Mark the reused register. It's not really clear if we support tied
// stack operands. We could do that for some Intel read-modify-write
// encodings.
if let ValueLoc::Reg(regunit) = loc {
// This is going to assert out unless the incoming value at
// `arg_index` was killed. Tied operands must be fixed to
// ensure that before running the coloring pass.
regs.take(opcst.regclass, regunit);
}
}
ConstraintKind::FixedReg(_regunit) => unimplemented!(),
ConstraintKind::Stack => {
panic!("{}:{} should be a stack value", lv.value, pref_rc.name)
}
}
}
Affinity::Stack => unimplemented!(),
Affinity::None => {
panic!("Encoded instruction defines {} with no affinity", lv.value)
}
}
// Program the fixed output constraints before the general defines. This allows us to
// detect conflicts between fixed outputs and tied operands where the input value hasn't
// been converted to a solver variable.
if constraints.fixed_outs {
self.program_fixed_output_constraints(inst,
constraints.outs,
defs,
throughs,
dfg,
locations);
}
self.program_output_constraints(inst, constraints.outs, defs, dfg, locations);
// Finally, we've fully programmed the constraint solver.
// We expect a quick solution in most cases.
let mut output_regs = self.solver
.quick_solve()
.unwrap_or_else(|_| self.iterate_solution());
// Apply the solution to the defs.
for v in self.solver.vars().iter().filter(|&v| v.is_define) {
*locations.ensure(v.value) = ValueLoc::Reg(v.solution);
}
// Get rid of the dead defs.
// Update `regs` for the next instruction, remove the dead defs.
for lv in defs {
if lv.endpoint == inst {
if let Affinity::Reg(rc_index) = lv.affinity {
let regclass = self.reginfo.rc(rc_index);
if let ValueLoc::Reg(regunit) = locations[lv.value] {
regs.free(regclass, regunit);
if let Affinity::Reg(rci) = lv.affinity {
let rc = self.reginfo.rc(rci);
let reg = self.divert.reg(lv.value, locations);
output_regs.free(rc, reg);
}
}
}
*regs = output_regs;
}
/// Program the input-side constraints for `inst` into the constraint solver.
fn program_input_constraints(&mut self,
inst: Inst,
constraints: &[OperandConstraint],
dfg: &mut DataFlowGraph,
locations: &mut EntityMap<Value, ValueLoc>) {
for (op, &value) in constraints
.iter()
.zip(dfg.inst_args(inst))
.filter(|&(op, _)| op.kind != ConstraintKind::Stack) {
// Reload pass is supposed to ensure that all arguments to register operands are
// already in a register.
let cur_reg = self.divert.reg(value, locations);
match op.kind {
ConstraintKind::FixedReg(regunit) => {
if regunit != cur_reg {
self.solver
.reassign_in(value, op.regclass, cur_reg, regunit);
}
}
ConstraintKind::Reg |
ConstraintKind::Tied(_) => {
if !op.regclass.contains(cur_reg) {
self.solver
.add_var(value, op.regclass, cur_reg, &self.reginfo);
}
}
ConstraintKind::Stack => unreachable!(),
}
}
}
// Find existing live values that conflict with the fixed input register constraints programmed
// into the constraint solver. Convert them to solver variables so they can be diverted.
fn divert_fixed_input_conflicts(&mut self,
live: &[LiveValue],
locations: &mut EntityMap<Value, ValueLoc>) {
for lv in live {
if let Affinity::Reg(rci) = lv.affinity {
let rc = self.reginfo.rc(rci);
let reg = self.divert.reg(lv.value, locations);
if self.solver.is_fixed_input_conflict(rc, reg) {
self.solver.add_var(lv.value, rc, reg, &self.reginfo);
}
}
}
}
/// Program any fixed-register output constraints into the solver. This may also detect
/// conflicts between live-through registers and fixed output registers. These live-through
/// values need to be turned into solver variables so they can be reassigned.
fn program_fixed_output_constraints(&mut self,
_inst: Inst,
constraints: &[OperandConstraint],
defs: &[LiveValue],
throughs: &[LiveValue],
_dfg: &mut DataFlowGraph,
locations: &mut EntityMap<Value, ValueLoc>) {
for (op, lv) in constraints.iter().zip(defs) {
if let ConstraintKind::FixedReg(reg) = op.kind {
self.add_fixed_output(lv.value, op.regclass, reg, throughs, locations);
}
}
}
/// Add a single fixed output value to the solver.
fn add_fixed_output(&mut self,
value: Value,
rc: RegClass,
reg: RegUnit,
throughs: &[LiveValue],
locations: &mut EntityMap<Value, ValueLoc>) {
if !self.solver.add_fixed_output(rc, reg) {
// The fixed output conflicts with some of the live-through registers.
for lv in throughs {
if let Affinity::Reg(rci) = lv.affinity {
let rc2 = self.reginfo.rc(rci);
let reg2 = self.divert.reg(lv.value, locations);
if regs_overlap(rc, reg, rc2, reg2) {
// This live-through value is interfering with the fixed output assignment.
// Convert it to a solver variable.
// TODO: Use a looser constraint than the affinity hint. Any allocatable
// register in the top-level register class would be OK. Maybe `add_var`
// should take both a preferred class and a required constraint class.
self.solver.add_var(lv.value, rc2, reg2, &self.reginfo);
}
}
}
let ok = self.solver.add_fixed_output(rc, reg);
assert!(ok, "Couldn't clear fixed output interference for {}", value);
}
*locations.ensure(value) = ValueLoc::Reg(reg);
}
/// Program the output-side constraints for `inst` into the constraint solver.
///
/// It is assumed that all fixed outputs have already been handled.
fn program_output_constraints(&mut self,
_inst: Inst,
constraints: &[OperandConstraint],
defs: &[LiveValue],
_dfg: &mut DataFlowGraph,
_locations: &mut EntityMap<Value, ValueLoc>) {
for (op, lv) in constraints.iter().zip(defs) {
match op.kind {
ConstraintKind::FixedReg(_) |
ConstraintKind::Stack => continue,
ConstraintKind::Reg => {
self.solver.add_def(lv.value, op.regclass);
}
ConstraintKind::Tied(_) => unimplemented!(),
}
}
}
/// Try harder to find a solution to the constraint problem since `quick_solve()` failed.
///
/// We may need to move more registers around before a solution is possible. Use an iterative
/// algorithm that adds one more variable until a solution can be found.
fn iterate_solution(&self) -> AllocatableSet {
unimplemented!();
}
}

1
lib/cretonne/src/regalloc/mod.rs
View File

@@ -11,6 +11,7 @@ pub mod coloring;
mod affinity;
mod context;
mod diversion;
mod solver;
pub use self::allocatable_set::AllocatableSet;
pub use self::context::Context;

536
lib/cretonne/src/regalloc/solver.rs Normal file
View File

@@ -0,0 +1,536 @@
//! Constraint solver for register coloring.
//!
//! The coloring phase of SSA-based register allocation is very simple in theory, but in practice
//! it is complicated by the various constraints imposed by individual instructions:
//!
//! - Call and return instructions have to satisfy ABI requirements for arguments and return
//! values.
//! - Values live across a call must be in a callee-saved register.
//! - Some instructions have operand constraints such as register sub-classes, fixed registers, or
//! tied operands.
//!
//! # The instruction register coloring problem
//!
//! The constraint solver addresses the problem of satisfying the constraints of a single
//! instruction. We have:
//!
//! - A set of values that are live in registers before the instruction, with current register
//! assignments. Some are used by the instruction, some are not.
//! - A subset of the live register values that are killed by the instruction.
//! - A set of new register values that are defined by the instruction.
//! The constraint solver addresses the problem of satisfying the constraints of a single
//! instruction. We have:
//!
//! - A set of values that are live in registers before the instruction, with current register
//! assignments. Some are used by the instruction, some are not.
//! - A subset of the live register values that are killed by the instruction.
//! - A set of new register values that are defined by the instruction.
//!
//! We are not concerned with stack values at all. The reload pass ensures that all values required
//! to be in a register by the instruction are already in a register.
//!
//! A solution to the register coloring problem consists of:
//!
//! - Register reassignment prescriptions for a subset of the live register values.
//! - Register assignments for the defined values.
//!
//! The solution ensures that when live registers are reassigned as prescribed before the
//! instruction, all its operand constraints are satisfied, and the definition assignments won't
//! conflict.
//!
//! # Register diversions and global interference
//!
//! We can divert register values temporarily to satisfy constraints, but we need to put the
//! values back into their originally assigned register locations before leaving the EBB.
//! Otherwise, values won't be in the right register at the entry point of other EBBs.
//!
//! Some values are *local*, and we don't need to worry about putting those values back since they
//! are not used in any other EBBs.
//!
//! When we assign register locations to defines, we are assigning both the register used locally
//! immediately after the instruction and the register used globally when the defined value is used
//! in a different EBB. We need to avoid interference both locally at the instruction and globally.
//!
//! We have multiple mappings of values to registers:
//!
//! 1. The initial local mapping before the instruction. This includes any diversions from previous
//! instructions in the EBB, but not diversions for the current instruction.
//! 2. The local mapping after applying the additional reassignments required to satisfy the
//! constraints of the current instruction.
//! 3. The local mapping after the instruction. This excludes values killed by the instruction and
//! includes values defined by the instruction.
//! 4. The global mapping after the instruction. This mapping only contains values with global live
//! ranges, and it does not include any diversions.
//!
//! All four mappings must be kept free of interference.
//!
//! # Problems handled by previous passes.
//!
//! The constraint solver can only reassign registers, it can't create spill code, so some
//! constraints are handled by earlier passes:
//!
//! - There will be enough free registers available for the defines. Ensuring this is the primary
//! purpose of the spilling phase.
//! - When the same value is used for multiple operands, the intersection of operand constraints is
//! non-empty. The spilling phase will insert copies to handle mutually incompatible constraints,
//! such as when the same value is bound to two different function arguments.
//! - Values bound to tied operands must be killed by the instruction. Also enforced by the
//! spiller.
//! - Values used by register operands are in registers, and values used by stack operands are in
//! stack slots. This is enforced by the reload pass.
//!
//! # Solver algorithm
//!
//! The goal of the solver is to satisfy the instruction constraints with a minimal number of
//! register assignments before the instruction.
//!
//! 1. Compute the set of values used by operands with a fixed register constraint that isn't
//! already satisfied. These are mandatory predetermined reassignments.
//! 2. Compute the set of values that don't satisfy their register class constraint. These are
//! mandatory reassignments that we need to solve.
//! 3. Add the set of defines to the set of variables computed in 2. Exclude defines tied to an
//! input operand since their value is pre-determined.
//!
//! The set of values computed in 2. and 3. are the *variables* for the solver. Given a set of
//! variables, we can also compute a set of allocatable registers by removing the variables from
//! the set of assigned registers before the instruction.
//!
//! 1. For each variable, compute its domain as the intersection of the allocatable registers and
//! its register class constraint.
//! 2. Sort the variables in order of increasing domain size.
//! 3. Search for a solution that assigns each variable a register from its domain without
//! interference between variables.
//!
//! If the search fails to find a solution, we may need to reassign more registers. Find an
//! appropriate candidate among the set of live register values, add it as a variable and start
//! over.
use ir::Value;
use isa::{RegInfo, RegClass, RegUnit};
use regalloc::allocatable_set::RegSetIter;
use sparse_map::{SparseMap, SparseMapValue};
use std::fmt;
use super::AllocatableSet;
/// A variable in the constraint problem.
///
/// Variables represent register values that can be assigned to any register unit within the
/// constraint register class. This includes live register values that can be reassigned to a new
/// register and values defined by the instruction which must be assigned to a register.
///
/// Besides satisfying the register class constraint, variables must also be mutually
/// non-interfering in up to three contexts:
///
/// 1. Input side live registers, after applying all the reassignments.
/// 2. Output side live registers, considering all the local register diversions.
/// 3. Global live register, not considering any local diversions.
///
pub struct Variable {
/// The value whose register assignment we're looking for.
pub value: Value,
/// Avoid interference on the input side.
is_input: bool,
/// Avoid interference on the output side.
is_output: bool,
/// Avoid interference with the global registers.
is_global: bool,
/// The value is defined by the current instruction.
pub is_define: bool,
/// Number of registers available in the domain of this variable.
domain: u16,
/// The assigned register unit after a full solution was found.
pub solution: RegUnit,
/// Any solution must belong to the constraint register class.
constraint: RegClass,
}
impl Variable {
/// Get an iterator over possible register choices, given the available registers on the input
/// and output sides respectively.
fn iter(&self, iregs: &AllocatableSet, oregs: &AllocatableSet) -> RegSetIter {
if self.is_input && self.is_output {
let mut r = iregs.clone();
r.intersect(oregs);
r.iter(self.constraint)
} else if self.is_input {
iregs.iter(self.constraint)
} else {
oregs.iter(self.constraint)
}
}
}
impl fmt::Display for Variable {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{}({}", self.value, self.constraint)?;
if self.is_input {
write!(f, ", in")?;
}
if self.is_output {
write!(f, ", out")?;
}
if self.is_global {
write!(f, ", global")?;
}
if self.is_define {
write!(f, ", def")?;
}
if self.domain > 0 {
write!(f, ", {}", self.domain)?;
}
write!(f, ")")
}
}
struct Assignment {
value: Value,
from: RegUnit,
to: RegUnit,
rc: RegClass,
}
impl SparseMapValue<Value> for Assignment {
fn key(&self) -> Value {
self.value
}
}
/// Constraint solver for register allocation around a single instruction.
///
/// Start by programming in the instruction constraints.
///
/// 1. Initialize the solver by calling `reset()` with the set of allocatable registers before the
/// instruction.
/// 2. Program the input side constraints: Call `reassign_in()` for all fixed register constraints,
/// and `add_var()` for any input operands whose constraints are not already satisfied.
/// 3. Check for conflicts between fixed input assignments and existing live values by calling
/// `has_fixed_input_conflicts()`. Resolve any conflicts by calling `add_var()` with the
/// conflicting values.
/// 4. Prepare for adding output side constraints by calling `inputs_done()`.
/// 5. Add any killed register values that no longer cause interference on the output side by
/// calling `add_kill()`.
/// 6. Program the output side constraints: Call `add_fixed_output()` for all fixed register
/// constraints and `add_def()` for free defines. Resolve fixed output conflicts by calling
/// `add_var()`.
///
pub struct Solver {
/// Register reassignments that are required or decided as part of a full solution.
assignments: SparseMap<Value, Assignment>,
/// Variables are the values that should be reassigned as part of a solution.
/// Values with a fixed register constraints are not considered variables. They are represented
/// in the `assignments` vector if necessary.
vars: Vec<Variable>,
/// Are we finished adding input-side constraints? This changes the meaning of the `regs_in`
/// and `regs_out` register sets.
inputs_done: bool,
/// Available registers on the input side of the instruction.
///
/// While we're adding input constraints (`!inputs_done`):
///
/// - Live values on the input side are marked as unavailable.
/// - The 'from' registers of fixed input reassignments are marked as available as they are
/// added.
/// - Input-side variables are marked as available.
///
/// After finishing input constraints (`inputs_done`):
///
/// - Live values on the input side are marked as unavailable.
/// - The 'to' registers of fixed input reassignments are marked as unavailable.
/// - Input-side variables are marked as available.
///
regs_in: AllocatableSet,
/// Available registers on the output side of the instruction / fixed input scratch space.
///
/// While we're adding input constraints (`!inputs_done`):
///
/// - The 'to' registers of fixed input reassignments are marked as unavailable.
///
/// After finishing input constraints (`inputs_done`):
///
/// - Live-through values are marked as unavailable.
/// - Fixed output assignments are marked as unavailable.
/// - Live-through variables are marked as available.
///
regs_out: AllocatableSet,
}
/// Interface for programming the constraints into the solver.
impl Solver {
/// Create a new empty solver.
pub fn new() -> Solver {
Solver {
assignments: SparseMap::new(),
vars: Vec::new(),
inputs_done: false,
regs_in: AllocatableSet::new(),
regs_out: AllocatableSet::new(),
}
}
/// Reset the solver state and prepare solving for a new instruction with an initial set of
/// allocatable registers.
///
/// The `regs` set is the allocatable registers before any reassignments are applied.
pub fn reset(&mut self, regs: &AllocatableSet) {
self.assignments.clear();
self.vars.clear();
self.inputs_done = false;
self.regs_in = regs.clone();
// Used for tracking fixed input assignments while `!inputs_done`:
self.regs_out = AllocatableSet::new();
}
/// Add a fixed input reassignment of `value`.
///
/// This means that `value` must be assigned to `to` and can't become a variable. Call with
/// `from == to` to ensure that `value` is not reassigned from its existing register location.
///
/// In either case, `to` will not be available for variables on the input side of the
/// instruction.
pub fn reassign_in(&mut self, value: Value, rc: RegClass, from: RegUnit, to: RegUnit) {
debug_assert!(!self.inputs_done);
if self.regs_in.is_avail(rc, from) {
// It looks like `value` was already removed from the register set. It must have been
// added as a variable previously. A fixed constraint beats a variable, so convert it.
if let Some(idx) = self.vars.iter().position(|v| v.value == value) {
let v = self.vars.remove(idx);
dbg!("Converting variable {} to a fixed constraint", v);
// The spiller is responsible for ensuring that all constraints on the uses of a
// value are compatible.
assert!(v.constraint.contains(to),
"Incompatible constraints for {}",
value);
} else {
panic!("Invalid from register for fixed {} constraint", value);
}
}
self.regs_in.free(rc, from);
self.regs_out.take(rc, to);
self.assignments
.insert(Assignment {
value,
rc,
from,
to,
});
}
/// Add a variable representing an input side value with an existing register assignment.
///
/// A variable is a value that should be reassigned to something in the `constraint` register
/// class.
///
/// It is assumed initially that the value is also live on the output side of the instruction.
/// This can be changed by calling to `add_kill()`.
pub fn add_var(&mut self,
value: Value,
constraint: RegClass,
from: RegUnit,
reginfo: &RegInfo) {
// Check for existing entries for this value.
if self.regs_in.is_avail(constraint, from) {
// There cold be an existing variable entry.
if let Some(v) = self.vars.iter_mut().find(|v| v.value == value) {
// We have an existing variable entry for `value`. Combine the constraints.
if let Some(rci) = v.constraint.intersect(constraint) {
v.constraint = reginfo.rc(rci);
return;
} else {
// The spiller should have made sure the same value is not used with disjoint
// constraints.
panic!("Incompatible constraints: {} + {}", constraint, *v)
}
}
// No variable, then it must be a fixed reassignment.
if let Some(a) = self.assignments.get(value) {
assert!(constraint.contains(a.to),
"Incompatible constraints for {}",
value);
return;
}
panic!("Wrong from register for {}", value);
}
self.regs_in.free(constraint, from);
if self.inputs_done {
self.regs_out.free(constraint, from);
}
self.vars
.push(Variable {
value,
constraint,
is_input: true,
is_output: true,
is_global: false,
is_define: false,
domain: 0,
solution: !0,
});
}
/// Check for conflicts between fixed input assignments and existing live values.
///
/// Returns true if one of the live values conflicts with a fixed input assignment. Such a
/// conflicting value must be turned into a variable.
pub fn has_fixed_input_conflicts(&self) -> bool {
debug_assert!(!self.inputs_done);
// The `from` side of the fixed input diversions are taken from `regs_out`.
self.regs_out.interferes_with(&self.regs_in)
}
/// Check if `rc, reg` specifically conflicts with the fixed input assignments.
pub fn is_fixed_input_conflict(&self, rc: RegClass, reg: RegUnit) -> bool {
debug_assert!(!self.inputs_done);
!self.regs_out.is_avail(rc, reg)
}
/// Finish adding input side constraints.
///
/// Call this method to indicate that there will be no more fixed input reassignments added
/// and prepare for the output side constraints.
pub fn inputs_done(&mut self) {
assert!(!self.has_fixed_input_conflicts());
// At this point, `regs_out` contains the `to` side of the input reassignments, and the
// `from` side has already been marked as available in `regs_in`.
//
// Remove the `to` assignments from `regs_in` so it now indicates the registers available
// to variables at the input side.
self.regs_in.intersect(&self.regs_out);
// The meaning of `regs_out` now changes completely to indicate the registers available to
// variables on the output side.
// The initial mask will be modified by `add_kill()` and `add_fixed_output()`.
self.regs_out = self.regs_in.clone();
// Now we can't add more fixed input assignments, but `add_var()` is still allowed.
self.inputs_done = true;
}
/// Record that an input register value is killed by the instruction.
///
/// Even if a fixed reassignment has been added for the value, the `reg` argument should be the
/// original location before the reassignments.
///
/// This means that the register is available on the output side.
pub fn add_kill(&mut self, value: Value, rc: RegClass, reg: RegUnit) {
debug_assert!(self.inputs_done);
// If a fixed assignment is killed, the `to` register becomes available on the output side.
if let Some(a) = self.assignments.get(value) {
debug_assert_eq!(a.from, reg);
self.regs_out.free(a.rc, a.to);
return;
}
// It's also possible that a variable is killed. That means it doesn't need to satisfy
// interference constraints on the output side.
// Variables representing tied operands will get their `is_output` flag set again later.
if let Some(v) = self.vars.iter_mut().find(|v| v.value == value) {
assert!(v.is_input);
v.is_output = false;
return;
}
// Alright, this is just a boring value being killed by the instruction. Just reclaim
// the assigned register.
self.regs_out.free(rc, reg);
}
/// Add a fixed output assignment.
///
/// This means that `to` will not be available for variables on the output side of the
/// instruction.
///
/// Returns `false` if a live value conflicts with `to`, so it couldn't be added. Find the
/// conflicting live-through value and turn it into a variable before calling this method
/// again.
#[allow(dead_code)]
pub fn add_fixed_output(&mut self, rc: RegClass, reg: RegUnit) -> bool {
debug_assert!(self.inputs_done);
if self.regs_out.is_avail(rc, reg) {
self.regs_out.take(rc, reg);
true
} else {
false
}
}
/// Add a defined output value.
///
/// This is similar to `add_var`, except the value doesn't have a prior register assignment.
pub fn add_def(&mut self, value: Value, constraint: RegClass) {
debug_assert!(self.inputs_done);
self.vars
.push(Variable {
value,
constraint,
is_input: false,
is_output: true,
is_global: false,
is_define: true,
domain: 0,
solution: !0,
});
}
}
/// Interface for searching for a solution.
impl Solver {
/// Try a quick-and-dirty solution.
///
/// This is expected to succeed for most instructions since the constraint problem is almost
/// always trivial.
///
/// Returns `true` is a solution was found.
pub fn quick_solve(&mut self) -> Result<AllocatableSet, RegClass> {
self.find_solution()
}
/// Search for a solution with the current list of variables.
///
/// If a solution was found, returns `Ok(regs)` with the set of available registers on the
/// output side after the solution. If no solution could be found, returns `Err(rc)` with the
/// constraint register class that needs more available registers.
fn find_solution(&mut self) -> Result<AllocatableSet, RegClass> {
// Available registers on the input and output sides respectively.
let mut iregs = self.regs_in.clone();
let mut oregs = self.regs_out.clone();
for v in &mut self.vars {
let rc = v.constraint;
let reg = match v.iter(&iregs, &oregs).next() {
None => return Err(rc),
Some(reg) => reg,
};
v.solution = reg;
if v.is_input {
iregs.take(rc, reg);
}
if v.is_output {
oregs.take(rc, reg);
}
}
Ok(oregs)
}
/// Get all the variables.
pub fn vars(&self) -> &[Variable] {
&self.vars
}
}