T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

x64: Take SIGFPE signals for divide traps (#6026)

Browse Source 
* x64: Take SIGFPE signals for divide traps

Prior to this commit Wasmtime would configure `avoid_div_traps=true`
unconditionally for Cranelift. This, for the division-based
instructions, would change emitted code to explicitly trap on trap
conditions instead of letting the `div` x86 instruction trap.

There's no specific reason for Wasmtime, however, to specifically avoid
traps in the `div` instruction. This means that the extra generated
branches on x86 aren't necessary since the `div` and `idiv` instructions
already trap for similar conditions as wasm requires.

This commit instead disables the `avoid_div_traps` setting for
Wasmtime's usage of Cranelift. Subsequently the codegen rules were
updated slightly:

* When `avoid_div_traps=true`, traps are no longer emitted for `div`
  instructions.
* The `udiv`/`urem` instructions now list their trap as divide-by-zero
  instead of integer overflow.
* The lowering for `sdiv` was updated to still explicitly check for zero
  but the integer overflow case is deferred to the instruction itself.
* The lowering of `srem` no longer checks for zero and the listed trap
  for the `div` instruction is a divide-by-zero.

This means that the codegen for `udiv` and `urem` no longer have any
branches. The codegen for `sdiv` removes one branch but keeps the
zero-check to differentiate the two kinds of traps. The codegen for
`srem` removes one branch but keeps the -1 check since the semantics of
`srem` mismatch with the semantics of `idiv` with a -1 divisor
(specifically for INT_MIN).

This is unlikely to have really all that much of a speedup but was
something I noticed during #6008 which seemed like it'd be good to clean
up. Plus Wasmtime's signal handling was already set up to catch
`SIGFPE`, it was just never firing.

* Remove the `avoid_div_traps` cranelift setting

With no known users currently removing this should be possible and helps
simplify the x64 backend.

* x64: GC more support for avoid_div_traps

Remove the `validate_sdiv_divisor*` pseudo-instructions and clean up
some of the ISLE rules now that `div` is allowed to itself trap
unconditionally.

* x64: Store div trap code in instruction itself

* Keep divisors in registers, not in memory

Don't accidentally fold multiple traps together

* Handle EXC_ARITHMETIC on macos

* Update emit tests

* Update winch and tests
This commit is contained in:
Alex Crichton
2023-03-15 19:18:45 -05:00
committed by GitHub
parent 5ff2824ebb
commit 5ae8575296
72 changed files with 505 additions and 2624 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
cranelift/codegen/src/isa/x64/inst/emit.rs
View File

@@ -400,7 +400,18 @@ pub(crate) fn emit(
emit_std_enc_enc(sink, prefix, opcode, 1, subopcode, enc_src, rex_flags)
}
Inst::Div { sign, divisor, .. } | Inst::Div8 { sign, divisor, .. } => {
Inst::Div {
sign,
trap,
divisor,
..
}
| Inst::Div8 {
sign,
trap,
divisor,
..
} => {
let divisor = divisor.clone().to_reg_mem().with_allocs(allocs);
let size = match inst {
Inst::Div {
@@ -438,7 +449,7 @@ pub(crate) fn emit(
OperandSize::Size64 => (0xF7, LegacyPrefixes::None),
};
sink.add_trap(TrapCode::IntegerDivisionByZero);
sink.add_trap(*trap);
let subopcode = match sign {
DivSignedness::Signed => 7,
@@ -613,6 +624,7 @@ pub(crate) fn emit(
let inst = match size {
OperandSize::Size8 => Inst::div8(
DivSignedness::Signed,
TrapCode::IntegerDivisionByZero,
RegMem::reg(divisor),
Gpr::new(regs::rax()).unwrap(),
Writable::from_reg(Gpr::new(regs::rax()).unwrap()),
@@ -620,6 +632,7 @@ pub(crate) fn emit(
_ => Inst::div(
size,
DivSignedness::Signed,
TrapCode::IntegerDivisionByZero,
RegMem::reg(divisor),
Gpr::new(regs::rax()).unwrap(),
Gpr::new(regs::rdx()).unwrap(),
@@ -632,55 +645,6 @@ pub(crate) fn emit(
sink.bind_label(done_label);
}
Inst::ValidateSdivDivisor {
dividend, divisor, ..
}
| Inst::ValidateSdivDivisor64 {
dividend, divisor, ..
} => {
let orig_inst = &inst;
let divisor = allocs.next(divisor.to_reg());
let dividend = allocs.next(dividend.to_reg());
let size = match inst {
Inst::ValidateSdivDivisor { size, .. } => *size,
_ => OperandSize::Size64,
};
// First trap if the divisor is zero
let inst = Inst::cmp_rmi_r(size, RegMemImm::imm(0), divisor);
inst.emit(&[], sink, info, state);
let inst = Inst::trap_if(CC::Z, TrapCode::IntegerDivisionByZero);
inst.emit(&[], sink, info, state);
// Now check if the divisor is -1. If it is then additionally
// check if the dividend is INT_MIN. If it isn't then jump to the
// end. If both conditions here are true then trap.
let inst = Inst::cmp_rmi_r(size, RegMemImm::imm(0xffffffff), divisor);
inst.emit(&[], sink, info, state);
let done = sink.get_label();
one_way_jmp(sink, CC::NZ, done);
let int_min = match orig_inst {
Inst::ValidateSdivDivisor64 { tmp, .. } => {
let tmp = allocs.next(tmp.to_reg().to_reg());
let inst = Inst::imm(size, i64::MIN as u64, Writable::from_reg(tmp));
inst.emit(&[], sink, info, state);
RegMemImm::reg(tmp)
}
_ => RegMemImm::imm(match size {
OperandSize::Size8 => 0x80,
OperandSize::Size16 => 0x8000,
OperandSize::Size32 => 0x80000000,
OperandSize::Size64 => unreachable!(),
}),
};
let inst = Inst::cmp_rmi_r(size, int_min, dividend);
inst.emit(&[], sink, info, state);
let inst = Inst::trap_if(CC::Z, TrapCode::IntegerOverflow);
inst.emit(&[], sink, info, state);
sink.bind_label(done);
}
Inst::Imm {
dst_size,
simm64,

18
cranelift/codegen/src/isa/x64/inst/emit_tests.rs
View File

@@ -1750,6 +1750,7 @@ fn test_x64_emit() {
Inst::div(
OperandSize::Size32,
DivSignedness::Signed,
TrapCode::IntegerDivisionByZero,
RegMem::reg(regs::rsi()),
Gpr::new(regs::rax()).unwrap(),
Gpr::new(regs::rdx()).unwrap(),
@@ -1757,12 +1758,13 @@ fn test_x64_emit() {
WritableGpr::from_reg(Gpr::new(regs::rdx()).unwrap()),
),
"F7FE",
"idiv %eax, %edx, %esi, %eax, %edx",
"idiv %eax, %edx, %esi, %eax, %edx ; trap=int_divz",
));
insns.push((
Inst::div(
OperandSize::Size64,
DivSignedness::Signed,
TrapCode::IntegerDivisionByZero,
RegMem::reg(regs::r15()),
Gpr::new(regs::rax()).unwrap(),
Gpr::new(regs::rdx()).unwrap(),
@@ -1770,12 +1772,13 @@ fn test_x64_emit() {
WritableGpr::from_reg(Gpr::new(regs::rdx()).unwrap()),
),
"49F7FF",
"idiv %rax, %rdx, %r15, %rax, %rdx",
"idiv %rax, %rdx, %r15, %rax, %rdx ; trap=int_divz",
));
insns.push((
Inst::div(
OperandSize::Size32,
DivSignedness::Unsigned,
TrapCode::IntegerDivisionByZero,
RegMem::reg(regs::r14()),
Gpr::new(regs::rax()).unwrap(),
Gpr::new(regs::rdx()).unwrap(),
@@ -1783,12 +1786,13 @@ fn test_x64_emit() {
WritableGpr::from_reg(Gpr::new(regs::rdx()).unwrap()),
),
"41F7F6",
"div %eax, %edx, %r14d, %eax, %edx",
"div %eax, %edx, %r14d, %eax, %edx ; trap=int_divz",
));
insns.push((
Inst::div(
OperandSize::Size64,
DivSignedness::Unsigned,
TrapCode::IntegerDivisionByZero,
RegMem::reg(regs::rdi()),
Gpr::new(regs::rax()).unwrap(),
Gpr::new(regs::rdx()).unwrap(),
@@ -1796,27 +1800,29 @@ fn test_x64_emit() {
WritableGpr::from_reg(Gpr::new(regs::rdx()).unwrap()),
),
"48F7F7",
"div %rax, %rdx, %rdi, %rax, %rdx",
"div %rax, %rdx, %rdi, %rax, %rdx ; trap=int_divz",
));
insns.push((
Inst::div8(
DivSignedness::Unsigned,
TrapCode::IntegerDivisionByZero,
RegMem::reg(regs::rax()),
Gpr::new(regs::rax()).unwrap(),
WritableGpr::from_reg(Gpr::new(regs::rax()).unwrap()),
),
"F6F0",
"div %al, %al, %al",
"div %al, %al, %al ; trap=int_divz",
));
insns.push((
Inst::div8(
DivSignedness::Unsigned,
TrapCode::IntegerDivisionByZero,
RegMem::reg(regs::rsi()),
Gpr::new(regs::rax()).unwrap(),
WritableGpr::from_reg(Gpr::new(regs::rax()).unwrap()),
),
"40F6F6",
"div %al, %sil, %al",
"div %al, %sil, %al ; trap=int_divz",
));
// ========================================================

52
cranelift/codegen/src/isa/x64/inst/mod.rs
View File

@@ -73,8 +73,6 @@ impl Inst {
| Inst::CallUnknown { .. }
| Inst::CheckedSRemSeq { .. }
| Inst::CheckedSRemSeq8 { .. }
| Inst::ValidateSdivDivisor { .. }
| Inst::ValidateSdivDivisor64 { .. }
| Inst::Cmove { .. }
| Inst::CmpRmiR { .. }
| Inst::CvtFloatToSintSeq { .. }
@@ -225,6 +223,7 @@ impl Inst {
pub(crate) fn div(
size: OperandSize,
sign: DivSignedness,
trap: TrapCode,
divisor: RegMem,
dividend_lo: Gpr,
dividend_hi: Gpr,
@@ -235,6 +234,7 @@ impl Inst {
Inst::Div {
size,
sign,
trap,
divisor: GprMem::new(divisor).unwrap(),
dividend_lo,
dividend_hi,
@@ -245,6 +245,7 @@ impl Inst {
pub(crate) fn div8(
sign: DivSignedness,
trap: TrapCode,
divisor: RegMem,
dividend: Gpr,
dst: WritableGpr,
@@ -252,6 +253,7 @@ impl Inst {
divisor.assert_regclass_is(RegClass::Int);
Inst::Div8 {
sign,
trap,
divisor: GprMem::new(divisor).unwrap(),
dividend,
dst,
@@ -548,10 +550,6 @@ impl Inst {
Inst::JmpUnknown { target }
}
pub(crate) fn trap_if(cc: CC, trap_code: TrapCode) -> Inst {
Inst::TrapIf { cc, trap_code }
}
/// Choose which instruction to use for loading a register value from memory. For loads smaller
/// than 64 bits, this method expects a way to extend the value (i.e. [ExtKind::SignExtend],
/// [ExtKind::ZeroExtend]); loads with no extension necessary will ignore this.
@@ -771,6 +769,7 @@ impl PrettyPrint for Inst {
Inst::Div {
size,
sign,
trap,
divisor,
dividend_lo,
dividend_hi,
@@ -785,7 +784,7 @@ impl PrettyPrint for Inst {
let dst_remainder =
pretty_print_reg(dst_remainder.to_reg().to_reg(), size.to_bytes(), allocs);
format!(
"{} {}, {}, {}, {}, {}",
"{} {}, {}, {}, {}, {} ; trap={trap}",
ljustify(match sign {
DivSignedness::Signed => "idiv".to_string(),
DivSignedness::Unsigned => "div".to_string(),
@@ -800,6 +799,7 @@ impl PrettyPrint for Inst {
Inst::Div8 {
sign,
trap,
divisor,
dividend,
dst,
@@ -808,7 +808,7 @@ impl PrettyPrint for Inst {
let dividend = pretty_print_reg(dividend.to_reg(), 1, allocs);
let dst = pretty_print_reg(dst.to_reg().to_reg(), 1, allocs);
format!(
"{} {dividend}, {divisor}, {dst}",
"{} {dividend}, {divisor}, {dst} ; trap={trap}",
ljustify(match sign {
DivSignedness::Signed => "idiv".to_string(),
DivSignedness::Unsigned => "div".to_string(),
@@ -874,27 +874,6 @@ impl PrettyPrint for Inst {
format!("checked_srem_seq {dividend}, {divisor}, {dst}")
}
Inst::ValidateSdivDivisor {
dividend,
divisor,
size,
} => {
let dividend = pretty_print_reg(dividend.to_reg(), size.to_bytes(), allocs);
let divisor = pretty_print_reg(divisor.to_reg(), size.to_bytes(), allocs);
format!("validate_sdiv_divisor {dividend}, {divisor}")
}
Inst::ValidateSdivDivisor64 {
dividend,
divisor,
tmp,
} => {
let dividend = pretty_print_reg(dividend.to_reg(), 8, allocs);
let divisor = pretty_print_reg(divisor.to_reg(), 8, allocs);
let tmp = pretty_print_reg(tmp.to_reg().to_reg(), 8, allocs);
format!("validate_sdiv_divisor {dividend}, {divisor} {tmp}")
}
Inst::SignExtendData { size, src, dst } => {
let src = pretty_print_reg(src.to_reg(), size.to_bytes(), allocs);
let dst = pretty_print_reg(dst.to_reg().to_reg(), size.to_bytes(), allocs);
@@ -1917,21 +1896,6 @@ fn x64_get_operands<F: Fn(VReg) -> VReg>(inst: &Inst, collector: &mut OperandCol
collector.reg_fixed_def(dst_hi.to_writable_reg(), regs::rdx());
src2.get_operands(collector);
}
Inst::ValidateSdivDivisor {
dividend, divisor, ..
} => {
collector.reg_use(divisor.to_reg());
collector.reg_use(dividend.to_reg());
}
Inst::ValidateSdivDivisor64 {
dividend,
divisor,
tmp,
} => {
collector.reg_use(divisor.to_reg());
collector.reg_use(dividend.to_reg());
collector.reg_early_def(tmp.to_writable_reg());
}
Inst::SignExtendData { size, src, dst } => {
match size {
OperandSize::Size8 => {