x64: Add more support for more AVX instructions (#5931)

* x64: Add a smattering of lowerings for `shuffle` specializations (#5930)

* x64: Add lowerings for `punpck{h,l}wd`

Add some special cases for `shuffle` for more specialized x86
instructions.

* x64: Add `shuffle` lowerings for `pshufd`

This commit adds special-cased lowerings for the x64 `shuffle`
instruction when the `pshufd` instruction alone is necessary. This is
possible when the shuffle immediate permutes 32-bit values within one of
the vector inputs of the `shuffle` instruction, but not both.

* x64: Add shuffle lowerings for `punpck{h,l}{q,}dq`

This adds specific permutations for some x86 instructions which
specifically interleave high/low bytes for 32 and 64-bit values. This
corresponds to the preexisting specific lowerings for interleaving 8 and
16-bit values.

* x64: Add `shuffle` lowerings for `shufps`

This commit adds targeted lowerings for the `shuffle` instruction that
match the pattern that `shufps` supports. The `shufps` instruction
selects two elements from the first vector and two elements from the
second vector which means while it's not generally applicable it should
still be more useful than the catch-all lowering of `shuffle`.

* x64: Add shuffle support for `pshuf{l,h}w`

This commit adds special lowering cases for these instructions which
permute 16-bit values within a 128-bit value either within the upper or
lower half of the 128-bit value.

* x64: Specialize `shuffle` with an all-zeros immediate

Instead of loading the all-zeros immediate from a rip-relative address
at the end of the function instead generate a zero with a `pxor`
instruction and then use `pshufb` to do the broadcast.

* Review comments

* x64: Add an AVX encoding for the `pshufd` instruction

This will benefit from lack of need for alignment vs the `pshufd`
instruction if working with a memory operand and additionally, as I've
just learned, this reduces dependencies between instructions because the
`v*` instructions zero the upper bits as opposed to preserving them
which could accidentally create false dependencies in the CPU between
instructions.

* x64: Add more support for AVX loads/stores

This commit adds VEX-encoded versions of instructions such as
`mov{ss,sd,upd,ups,dqu}` for load and store operations. This also
changes some signatures so the `load` helpers specifically take a
`SyntheticAmode` argument which ended up doing a small refactoring of
the `*_regmove` variant used for `insertlane 0` into f64x2 vectors.

* x64: Enable using AVX instructions for zero regs

This commit refactors the internal ISLE helpers for creating zero'd
xmm registers to leverage the AVX support for all other instructions.
This moves away from picking opcodes to instead picking instructions
with a bit of reorganization.

* x64: Remove `XmmConstOp` as an instruction

All existing users can be replaced with usage of the `xmm_uninit_value`
helper instruction so there's no longer any need for these otherwise
constant operations. This additionally reduces manual usage of opcodes
in favor of instruction helpers.

* Review comments

* Update test expectations
This commit is contained in:
Alex Crichton
2023-03-09 17:57:42 -06:00
committed by GitHub
parent 1c3a1bda6c
commit 83f21e784a
22 changed files with 635 additions and 300 deletions

View File

@@ -69,9 +69,10 @@ block0(v0: f32x4):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pcmpeqd %xmm3, %xmm3, %xmm3
; psrld %xmm3, $1, %xmm3
; andps %xmm0, %xmm3, %xmm0
; uninit %xmm4
; pcmpeqd %xmm4, %xmm4, %xmm4
; psrld %xmm4, $1, %xmm4
; andps %xmm0, %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -81,9 +82,9 @@ block0(v0: f32x4):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pcmpeqd %xmm3, %xmm3
; psrld $1, %xmm3
; andps %xmm3, %xmm0
; pcmpeqd %xmm4, %xmm4
; psrld $1, %xmm4
; andps %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq
@@ -98,9 +99,10 @@ block0(v0: f64x2):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pcmpeqd %xmm3, %xmm3, %xmm3
; psrlq %xmm3, $1, %xmm3
; andpd %xmm0, %xmm3, %xmm0
; uninit %xmm4
; pcmpeqd %xmm4, %xmm4, %xmm4
; psrlq %xmm4, $1, %xmm4
; andpd %xmm0, %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -110,9 +112,9 @@ block0(v0: f64x2):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pcmpeqd %xmm3, %xmm3
; psrlq $1, %xmm3
; andpd %xmm3, %xmm0
; pcmpeqd %xmm4, %xmm4
; psrlq $1, %xmm4
; andpd %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -1032,20 +1032,22 @@ block0(v0: f32x4):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; xorps %xmm5, %xmm5, %xmm5
; movdqa %xmm0, %xmm9
; maxps %xmm9, %xmm5, %xmm9
; pcmpeqd %xmm5, %xmm5, %xmm5
; psrld %xmm5, $1, %xmm5
; cvtdq2ps %xmm5, %xmm13
; cvttps2dq %xmm9, %xmm12
; subps %xmm9, %xmm13, %xmm9
; cmpps $2, %xmm13, %xmm9, %xmm13
; cvttps2dq %xmm9, %xmm0
; pxor %xmm0, %xmm13, %xmm0
; pxor %xmm6, %xmm6, %xmm6
; pmaxsd %xmm0, %xmm6, %xmm0
; paddd %xmm0, %xmm12, %xmm0
; uninit %xmm6
; xorps %xmm6, %xmm6, %xmm6
; movdqa %xmm0, %xmm11
; maxps %xmm11, %xmm6, %xmm11
; pcmpeqd %xmm6, %xmm6, %xmm6
; psrld %xmm6, $1, %xmm6
; cvtdq2ps %xmm6, %xmm15
; cvttps2dq %xmm11, %xmm14
; subps %xmm11, %xmm15, %xmm11
; cmpps $2, %xmm15, %xmm11, %xmm15
; cvttps2dq %xmm11, %xmm0
; pxor %xmm0, %xmm15, %xmm0
; uninit %xmm9
; pxor %xmm9, %xmm9, %xmm9
; pmaxsd %xmm0, %xmm9, %xmm0
; paddd %xmm0, %xmm14, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -1055,20 +1057,20 @@ block0(v0: f32x4):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; xorps %xmm5, %xmm5
; movdqa %xmm0, %xmm9
; maxps %xmm5, %xmm9
; pcmpeqd %xmm5, %xmm5
; psrld $1, %xmm5
; cvtdq2ps %xmm5, %xmm13
; cvttps2dq %xmm9, %xmm12
; subps %xmm13, %xmm9
; cmpleps %xmm9, %xmm13
; cvttps2dq %xmm9, %xmm0
; pxor %xmm13, %xmm0
; pxor %xmm6, %xmm6
; pmaxsd %xmm6, %xmm0
; paddd %xmm12, %xmm0
; xorps %xmm6, %xmm6
; movdqa %xmm0, %xmm11
; maxps %xmm6, %xmm11
; pcmpeqd %xmm6, %xmm6
; psrld $1, %xmm6
; cvtdq2ps %xmm6, %xmm15
; cvttps2dq %xmm11, %xmm14
; subps %xmm15, %xmm11
; cmpleps %xmm11, %xmm15
; cvttps2dq %xmm11, %xmm0
; pxor %xmm15, %xmm0
; pxor %xmm9, %xmm9
; pmaxsd %xmm9, %xmm0
; paddd %xmm14, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -589,3 +589,59 @@ block0(v0: f64x2):
; addb %al, (%rax)
; sarb $0xff, %bh
function %load_and_store_f32(i64, i64) {
block0(v0: i64, v1: i64):
v2 = load.f32 v0
store v2, v1
return
}
; VCode:
; pushq %rbp
; movq %rsp, %rbp
; block0:
; vmovss 0(%rdi), %xmm3
; vmovss %xmm3, 0(%rsi)
; movq %rbp, %rsp
; popq %rbp
; ret
;
; Disassembled:
; block0: ; offset 0x0
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; vmovss (%rdi), %xmm3 ; trap: heap_oob
; vmovss %xmm3, (%rsi) ; trap: heap_oob
; movq %rbp, %rsp
; popq %rbp
; retq
function %load_and_store_f64(i64, i64) {
block0(v0: i64, v1: i64):
v2 = load.f64 v0
store v2, v1
return
}
; VCode:
; pushq %rbp
; movq %rsp, %rbp
; block0:
; vmovsd 0(%rdi), %xmm3
; vmovsd %xmm3, 0(%rsi)
; movq %rbp, %rsp
; popq %rbp
; ret
;
; Disassembled:
; block0: ; offset 0x0
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; vmovsd (%rdi), %xmm3 ; trap: heap_oob
; vmovsd %xmm3, (%rsi) ; trap: heap_oob
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -69,9 +69,10 @@ block0(v0: f32x4):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pcmpeqd %xmm3, %xmm3, %xmm3
; pslld %xmm3, $31, %xmm3
; xorps %xmm0, %xmm3, %xmm0
; uninit %xmm4
; pcmpeqd %xmm4, %xmm4, %xmm4
; pslld %xmm4, $31, %xmm4
; xorps %xmm0, %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -81,9 +82,9 @@ block0(v0: f32x4):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pcmpeqd %xmm3, %xmm3
; pslld $0x1f, %xmm3
; xorps %xmm3, %xmm0
; pcmpeqd %xmm4, %xmm4
; pslld $0x1f, %xmm4
; xorps %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq
@@ -98,9 +99,10 @@ block0(v0: f64x2):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pcmpeqd %xmm3, %xmm3, %xmm3
; psllq %xmm3, $63, %xmm3
; xorpd %xmm0, %xmm3, %xmm0
; uninit %xmm4
; pcmpeqd %xmm4, %xmm4, %xmm4
; psllq %xmm4, $63, %xmm4
; xorpd %xmm0, %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -110,9 +112,9 @@ block0(v0: f64x2):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pcmpeqd %xmm3, %xmm3
; psllq $0x3f, %xmm3
; xorpd %xmm3, %xmm0
; pcmpeqd %xmm4, %xmm4
; psllq $0x3f, %xmm4
; xorpd %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -0,0 +1,82 @@
test compile precise-output
set enable_simd
target x86_64 has_avx
function %insertlane_f64x2_zero(f64x2, f64) -> f64x2 {
block0(v0: f64x2, v1: f64):
v2 = insertlane v0, v1, 0
return v2
}
; VCode:
; pushq %rbp
; movq %rsp, %rbp
; block0:
; vmovsd %xmm0, %xmm1, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
;
; Disassembled:
; block0: ; offset 0x0
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; vmovsd %xmm1, %xmm0, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq
function %insertlane_f64x2_one(f64x2, f64) -> f64x2 {
block0(v0: f64x2, v1: f64):
v2 = insertlane v0, v1, 1
return v2
}
; VCode:
; pushq %rbp
; movq %rsp, %rbp
; block0:
; vmovlhps %xmm0, %xmm1, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
;
; Disassembled:
; block0: ; offset 0x0
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; vmovlhps %xmm1, %xmm0, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq
function %insertlane_f64x2_zero_with_load(f64x2, i64) -> f64x2 {
block0(v0: f64x2, v1: i64):
v2 = load.f64 v1
v3 = insertlane v0, v2, 0
return v3
}
; VCode:
; pushq %rbp
; movq %rsp, %rbp
; block0:
; vmovsd 0(%rdi), %xmm3
; vmovsd %xmm0, %xmm3, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
;
; Disassembled:
; block0: ; offset 0x0
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; vmovsd (%rdi), %xmm3 ; trap: heap_oob
; vmovsd %xmm3, %xmm0, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -626,8 +626,9 @@ block0(v0: i8x16, v1: i8x16):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pxor %xmm3, %xmm3, %xmm3
; pshufb %xmm0, %xmm3, %xmm0
; uninit %xmm4
; pxor %xmm4, %xmm4, %xmm4
; pshufb %xmm0, %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -637,8 +638,8 @@ block0(v0: i8x16, v1: i8x16):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pxor %xmm3, %xmm3
; pshufb %xmm3, %xmm0
; pxor %xmm4, %xmm4
; pshufb %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -610,8 +610,8 @@ block0(v0: i32x4, v1: i32x4):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pshufd $250, %xmm0, %xmm3
; pshufd $250, %xmm1, %xmm5
; vpshufd $250, %xmm0, %xmm3
; vpshufd $250, %xmm1, %xmm5
; vpmuldq %xmm3, %xmm5, %xmm0
; movq %rbp, %rsp
; popq %rbp
@@ -622,8 +622,8 @@ block0(v0: i32x4, v1: i32x4):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pshufd $0xfa, %xmm0, %xmm3
; pshufd $0xfa, %xmm1, %xmm5
; vpshufd $0xfa, %xmm0, %xmm3
; vpshufd $0xfa, %xmm1, %xmm5
; vpmuldq %xmm5, %xmm3, %xmm0
; movq %rbp, %rsp
; popq %rbp
@@ -641,8 +641,8 @@ block0(v0: i32x4, v1: i32x4):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pshufd $80, %xmm0, %xmm3
; pshufd $80, %xmm1, %xmm5
; vpshufd $80, %xmm0, %xmm3
; vpshufd $80, %xmm1, %xmm5
; vpmuludq %xmm3, %xmm5, %xmm0
; movq %rbp, %rsp
; popq %rbp
@@ -653,8 +653,8 @@ block0(v0: i32x4, v1: i32x4):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pshufd $0x50, %xmm0, %xmm3
; pshufd $0x50, %xmm1, %xmm5
; vpshufd $0x50, %xmm0, %xmm3
; vpshufd $0x50, %xmm1, %xmm5
; vpmuludq %xmm5, %xmm3, %xmm0
; movq %rbp, %rsp
; popq %rbp
@@ -1233,7 +1233,7 @@ block0(v0: i8x16):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; movdqu const(0), %xmm2
; vmovdqu const(0), %xmm2
; vpmaddubsw %xmm2, %xmm0, %xmm0
; movq %rbp, %rsp
; popq %rbp
@@ -1244,7 +1244,7 @@ block0(v0: i8x16):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; movdqu 0x14(%rip), %xmm2
; vmovdqu 0x14(%rip), %xmm2
; vpmaddubsw %xmm0, %xmm2, %xmm0
; movq %rbp, %rsp
; popq %rbp
@@ -1317,8 +1317,9 @@ block0(v0: i8):
; block0:
; uninit %xmm2
; vpinsrb $0 %xmm2, %rdi, %xmm4
; pxor %xmm6, %xmm6, %xmm6
; vpshufb %xmm4, %xmm6, %xmm0
; uninit %xmm6
; vpxor %xmm6, %xmm6, %xmm8
; vpshufb %xmm4, %xmm8, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -1329,8 +1330,8 @@ block0(v0: i8):
; movq %rsp, %rbp
; block1: ; offset 0x4
; vpinsrb $0, %edi, %xmm2, %xmm4
; pxor %xmm6, %xmm6
; vpshufb %xmm6, %xmm4, %xmm0
; vpxor %xmm6, %xmm6, %xmm8
; vpshufb %xmm8, %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq
@@ -1347,12 +1348,13 @@ block0(v0: f64x2):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; xorpd %xmm2, %xmm2, %xmm2
; vmaxpd %xmm0, %xmm2, %xmm4
; vminpd %xmm4, const(0), %xmm6
; vroundpd $3, %xmm6, %xmm8
; vaddpd %xmm8, const(1), %xmm10
; vshufps $136 %xmm10, %xmm2, %xmm0
; uninit %xmm2
; vxorpd %xmm2, %xmm2, %xmm4
; vmaxpd %xmm0, %xmm4, %xmm6
; vminpd %xmm6, const(0), %xmm8
; vroundpd $3, %xmm8, %xmm10
; vaddpd %xmm10, const(1), %xmm12
; vshufps $136 %xmm12, %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -1362,12 +1364,12 @@ block0(v0: f64x2):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; xorpd %xmm2, %xmm2
; vmaxpd %xmm2, %xmm0, %xmm4
; vminpd 0x1c(%rip), %xmm4, %xmm6
; vroundpd $3, %xmm6, %xmm8
; vaddpd 0x1e(%rip), %xmm8, %xmm10
; vshufps $0x88, %xmm2, %xmm10, %xmm0
; vxorpd %xmm2, %xmm2, %xmm4
; vmaxpd %xmm4, %xmm0, %xmm6
; vminpd 0x1c(%rip), %xmm6, %xmm8
; vroundpd $3, %xmm8, %xmm10
; vaddpd 0x1e(%rip), %xmm10, %xmm12
; vshufps $0x88, %xmm4, %xmm12, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq
@@ -1392,7 +1394,7 @@ block0(v0: i8x16, v1: i32):
; vpsllw %xmm0, %xmm5, %xmm7
; lea const(0), %rsi
; shlq $4, %r10, %r10
; movdqu 0(%rsi,%r10,1), %xmm13
; vmovdqu 0(%rsi,%r10,1), %xmm13
; vpand %xmm7, %xmm13, %xmm0
; movq %rbp, %rsp
; popq %rbp
@@ -1409,7 +1411,7 @@ block0(v0: i8x16, v1: i32):
; vpsllw %xmm5, %xmm0, %xmm7
; leaq 0x15(%rip), %rsi
; shlq $4, %r10
; movdqu (%rsi, %r10), %xmm13
; vmovdqu (%rsi, %r10), %xmm13
; vpand %xmm13, %xmm7, %xmm0
; movq %rbp, %rsp
; popq %rbp
@@ -1427,7 +1429,7 @@ block0(v0: i8x16):
; movq %rsp, %rbp
; block0:
; vpsllw %xmm0, $1, %xmm2
; movdqu const(0), %xmm4
; vmovdqu const(0), %xmm4
; vpand %xmm2, %xmm4, %xmm0
; movq %rbp, %rsp
; popq %rbp
@@ -1439,7 +1441,7 @@ block0(v0: i8x16):
; movq %rsp, %rbp
; block1: ; offset 0x4
; vpsllw $1, %xmm0, %xmm2
; movdqu 0xf(%rip), %xmm4
; vmovdqu 0xf(%rip), %xmm4
; vpand %xmm4, %xmm2, %xmm0
; movq %rbp, %rsp
; popq %rbp

View File

@@ -187,9 +187,10 @@ block0(v0: f32x4):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pcmpeqd %xmm2, %xmm2, %xmm2
; vpsrld %xmm2, $1, %xmm4
; vandps %xmm0, %xmm4, %xmm0
; uninit %xmm2
; vpcmpeqd %xmm2, %xmm2, %xmm4
; vpsrld %xmm4, $1, %xmm6
; vandps %xmm0, %xmm6, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -199,9 +200,9 @@ block0(v0: f32x4):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pcmpeqd %xmm2, %xmm2
; vpsrld $1, %xmm2, %xmm4
; vandps %xmm4, %xmm0, %xmm0
; vpcmpeqd %xmm2, %xmm2, %xmm4
; vpsrld $1, %xmm4, %xmm6
; vandps %xmm6, %xmm0, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -13,8 +13,9 @@ block0(v0: i32x4, v1: i32x4):
; movq %rsp, %rbp
; block0:
; pcmpeqd %xmm0, %xmm1, %xmm0
; pcmpeqd %xmm5, %xmm5, %xmm5
; pxor %xmm0, %xmm5, %xmm0
; uninit %xmm6
; pcmpeqd %xmm6, %xmm6, %xmm6
; pxor %xmm0, %xmm6, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -25,8 +26,8 @@ block0(v0: i32x4, v1: i32x4):
; movq %rsp, %rbp
; block1: ; offset 0x4
; pcmpeqd %xmm1, %xmm0
; pcmpeqd %xmm5, %xmm5
; pxor %xmm5, %xmm0
; pcmpeqd %xmm6, %xmm6
; pxor %xmm6, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq
@@ -43,8 +44,9 @@ block0(v0: i32x4, v1: i32x4):
; block0:
; pmaxud %xmm0, %xmm1, %xmm0
; pcmpeqd %xmm0, %xmm1, %xmm0
; pcmpeqd %xmm7, %xmm7, %xmm7
; pxor %xmm0, %xmm7, %xmm0
; uninit %xmm8
; pcmpeqd %xmm8, %xmm8, %xmm8
; pxor %xmm0, %xmm8, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -56,8 +58,8 @@ block0(v0: i32x4, v1: i32x4):
; block1: ; offset 0x4
; pmaxud %xmm1, %xmm0
; pcmpeqd %xmm1, %xmm0
; pcmpeqd %xmm7, %xmm7
; pxor %xmm7, %xmm0
; pcmpeqd %xmm8, %xmm8
; pxor %xmm8, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -172,8 +172,9 @@ block0(v0: i8):
; block0:
; uninit %xmm0
; pinsrb $0, %xmm0, %rdi, %xmm0
; pxor %xmm6, %xmm6, %xmm6
; pshufb %xmm0, %xmm6, %xmm0
; uninit %xmm7
; pxor %xmm7, %xmm7, %xmm7
; pshufb %xmm0, %xmm7, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -184,8 +185,8 @@ block0(v0: i8):
; movq %rsp, %rbp
; block1: ; offset 0x4
; pinsrb $0, %edi, %xmm0
; pxor %xmm6, %xmm6
; pshufb %xmm6, %xmm0
; pxor %xmm7, %xmm7
; pshufb %xmm7, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -152,3 +152,87 @@ block0(v0: i64):
; popq %rbp
; retq
function %load_store_i8x16(i64, i64) {
block0(v0: i64, v1: i64):
v2 = load.i8x16 v0
store v2, v1
return
}
; VCode:
; pushq %rbp
; movq %rsp, %rbp
; block0:
; vmovdqu 0(%rdi), %xmm3
; vmovdqu %xmm3, 0(%rsi)
; movq %rbp, %rsp
; popq %rbp
; ret
;
; Disassembled:
; block0: ; offset 0x0
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; vmovdqu (%rdi), %xmm3 ; trap: heap_oob
; vmovdqu %xmm3, (%rsi) ; trap: heap_oob
; movq %rbp, %rsp
; popq %rbp
; retq
function %load_store_f32x4(i64, i64) {
block0(v0: i64, v1: i64):
v2 = load.f32x4 v0
store v2, v1
return
}
; VCode:
; pushq %rbp
; movq %rsp, %rbp
; block0:
; vmovups 0(%rdi), %xmm3
; vmovups %xmm3, 0(%rsi)
; movq %rbp, %rsp
; popq %rbp
; ret
;
; Disassembled:
; block0: ; offset 0x0
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; vmovups (%rdi), %xmm3 ; trap: heap_oob
; vmovups %xmm3, (%rsi) ; trap: heap_oob
; movq %rbp, %rsp
; popq %rbp
; retq
function %load_store_f64x2(i64, i64) {
block0(v0: i64, v1: i64):
v2 = load.f64x2 v0
store v2, v1
return
}
; VCode:
; pushq %rbp
; movq %rsp, %rbp
; block0:
; vmovupd 0(%rdi), %xmm3
; vmovupd %xmm3, 0(%rsi)
; movq %rbp, %rsp
; popq %rbp
; ret
;
; Disassembled:
; block0: ; offset 0x0
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; vmovupd (%rdi), %xmm3 ; trap: heap_oob
; vmovupd %xmm3, (%rsi) ; trap: heap_oob
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -12,8 +12,9 @@ block0(v0: i32x4):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pcmpeqd %xmm2, %xmm2, %xmm2
; pxor %xmm0, %xmm2, %xmm0
; uninit %xmm3
; pcmpeqd %xmm3, %xmm3, %xmm3
; pxor %xmm0, %xmm3, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -23,8 +24,8 @@ block0(v0: i32x4):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pcmpeqd %xmm2, %xmm2
; pxor %xmm2, %xmm0
; pcmpeqd %xmm3, %xmm3
; pxor %xmm3, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq
@@ -66,10 +67,11 @@ block0(v0: i64x2):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; pxor %xmm2, %xmm2, %xmm2
; movdqa %xmm0, %xmm4
; pcmpeqq %xmm4, %xmm2, %xmm4
; ptest %xmm4, %xmm4
; uninit %xmm3
; pxor %xmm3, %xmm3, %xmm3
; movdqa %xmm0, %xmm6
; pcmpeqq %xmm6, %xmm3, %xmm6
; ptest %xmm6, %xmm6
; setz %al
; movq %rbp, %rsp
; popq %rbp
@@ -80,10 +82,10 @@ block0(v0: i64x2):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; pxor %xmm2, %xmm2
; movdqa %xmm0, %xmm4
; pcmpeqq %xmm2, %xmm4
; ptest %xmm4, %xmm4
; pxor %xmm3, %xmm3
; movdqa %xmm0, %xmm6
; pcmpeqq %xmm3, %xmm6
; ptest %xmm6, %xmm6
; sete %al
; movq %rbp, %rsp
; popq %rbp

View File

@@ -13,13 +13,14 @@ block0(v0: f64x2):
; pushq %rbp
; movq %rsp, %rbp
; block0:
; xorpd %xmm2, %xmm2, %xmm2
; movdqa %xmm0, %xmm5
; maxpd %xmm5, %xmm2, %xmm5
; minpd %xmm5, const(0), %xmm5
; roundpd $3, %xmm5, %xmm0
; uninit %xmm3
; xorpd %xmm3, %xmm3, %xmm3
; movdqa %xmm0, %xmm7
; maxpd %xmm7, %xmm3, %xmm7
; minpd %xmm7, const(0), %xmm7
; roundpd $3, %xmm7, %xmm0
; addpd %xmm0, const(1), %xmm0
; shufps $136, %xmm0, %xmm2, %xmm0
; shufps $136, %xmm0, %xmm3, %xmm0
; movq %rbp, %rsp
; popq %rbp
; ret
@@ -29,13 +30,13 @@ block0(v0: f64x2):
; pushq %rbp
; movq %rsp, %rbp
; block1: ; offset 0x4
; xorpd %xmm2, %xmm2
; movdqa %xmm0, %xmm5
; maxpd %xmm2, %xmm5
; minpd 0x18(%rip), %xmm5
; roundpd $3, %xmm5, %xmm0
; xorpd %xmm3, %xmm3
; movdqa %xmm0, %xmm7
; maxpd %xmm3, %xmm7
; minpd 0x18(%rip), %xmm7
; roundpd $3, %xmm7, %xmm0
; addpd 0x1a(%rip), %xmm0
; shufps $0x88, %xmm2, %xmm0
; shufps $0x88, %xmm3, %xmm0
; movq %rbp, %rsp
; popq %rbp
; retq

View File

@@ -4,6 +4,7 @@ target aarch64
target s390x
set enable_simd
target x86_64 has_sse3 has_ssse3 has_sse41
target x86_64 has_sse3 has_ssse3 has_sse41 has_avx
function %insertlane_15(i8x16, i8) -> i8x16 {
block0(v0: i8x16, v1: i8):
@@ -32,3 +33,17 @@ block0(v0: i64x2, v1: i64):
return v2
}
; run: %insertlane_0([1 1], 5000000000) == [5000000000 1]
function %insertlane_0_in_f64x2(f64x2, f64) -> f64x2 {
block0(v0: f64x2, v1: f64):
v2 = insertlane v0, v1, 0
return v2
}
; run: %insertlane_0_in_f64x2([0x1.0 0x2.0], 0x3.0) == [0x3.0 0x2.0]
function %insertlane_1_in_f64x2(f64x2, f64) -> f64x2 {
block0(v0: f64x2, v1: f64):
v2 = insertlane v0, v1, 1
return v2
}
; run: %insertlane_1_in_f64x2([0x1.0 0x2.0], 0x3.0) == [0x1.0 0x3.0]

View File

@@ -63,19 +63,21 @@
;; movq %rsp, %rbp
;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 }
;; block0:
;; xorps %xmm3, %xmm3, %xmm3
;; vmaxps %xmm0, %xmm3, %xmm5
;; vpcmpeqd %xmm3, %xmm3, %xmm7
;; vpsrld %xmm7, $1, %xmm9
;; vcvtdq2ps %xmm9, %xmm11
;; vcvttps2dq %xmm5, %xmm13
;; vsubps %xmm5, %xmm11, %xmm15
;; vcmpps $2 %xmm11, %xmm15, %xmm1
;; vcvttps2dq %xmm15, %xmm3
;; vpxor %xmm3, %xmm1, %xmm5
;; pxor %xmm7, %xmm7, %xmm7
;; vpmaxsd %xmm5, %xmm7, %xmm9
;; vpaddd %xmm9, %xmm13, %xmm0
;; uninit %xmm3
;; vxorps %xmm3, %xmm3, %xmm5
;; vmaxps %xmm0, %xmm5, %xmm7
;; vpcmpeqd %xmm5, %xmm5, %xmm9
;; vpsrld %xmm9, $1, %xmm11
;; vcvtdq2ps %xmm11, %xmm13
;; vcvttps2dq %xmm7, %xmm15
;; vsubps %xmm7, %xmm13, %xmm1
;; vcmpps $2 %xmm13, %xmm1, %xmm3
;; vcvttps2dq %xmm1, %xmm5
;; vpxor %xmm5, %xmm3, %xmm7
;; uninit %xmm9
;; vpxor %xmm9, %xmm9, %xmm11
;; vpmaxsd %xmm7, %xmm11, %xmm13
;; vpaddd %xmm13, %xmm15, %xmm0
;; jmp label1
;; block1:
;; movq %rbp, %rsp
@@ -104,12 +106,13 @@
;; movq %rsp, %rbp
;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 }
;; block0:
;; xorpd %xmm3, %xmm3, %xmm3
;; vmaxpd %xmm0, %xmm3, %xmm5
;; vminpd %xmm5, const(0), %xmm7
;; vroundpd $3, %xmm7, %xmm9
;; vaddpd %xmm9, const(1), %xmm11
;; vshufps $136 %xmm11, %xmm3, %xmm0
;; uninit %xmm3
;; vxorpd %xmm3, %xmm3, %xmm5
;; vmaxpd %xmm0, %xmm5, %xmm7
;; vminpd %xmm7, const(0), %xmm9
;; vroundpd $3, %xmm9, %xmm11
;; vaddpd %xmm11, const(1), %xmm13
;; vshufps $136 %xmm13, %xmm5, %xmm0
;; jmp label1
;; block1:
;; movq %rbp, %rsp

View File

@@ -55,20 +55,22 @@
;; movq %rsp, %rbp
;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 }
;; block0:
;; xorps %xmm6, %xmm6, %xmm6
;; movdqa %xmm0, %xmm10
;; maxps %xmm10, %xmm6, %xmm10
;; pcmpeqd %xmm6, %xmm6, %xmm6
;; psrld %xmm6, $1, %xmm6
;; cvtdq2ps %xmm6, %xmm14
;; cvttps2dq %xmm10, %xmm13
;; subps %xmm10, %xmm14, %xmm10
;; cmpps $2, %xmm14, %xmm10, %xmm14
;; cvttps2dq %xmm10, %xmm0
;; pxor %xmm0, %xmm14, %xmm0
;; pxor %xmm7, %xmm7, %xmm7
;; pmaxsd %xmm0, %xmm7, %xmm0
;; paddd %xmm0, %xmm13, %xmm0
;; uninit %xmm7
;; xorps %xmm7, %xmm7, %xmm7
;; movdqa %xmm0, %xmm12
;; maxps %xmm12, %xmm7, %xmm12
;; pcmpeqd %xmm7, %xmm7, %xmm7
;; psrld %xmm7, $1, %xmm7
;; cvtdq2ps %xmm7, %xmm1
;; cvttps2dq %xmm12, %xmm15
;; subps %xmm12, %xmm1, %xmm12
;; cmpps $2, %xmm1, %xmm12, %xmm1
;; cvttps2dq %xmm12, %xmm0
;; pxor %xmm0, %xmm1, %xmm0
;; uninit %xmm10
;; pxor %xmm10, %xmm10, %xmm10
;; pmaxsd %xmm0, %xmm10, %xmm0
;; paddd %xmm0, %xmm15, %xmm0
;; jmp label1
;; block1:
;; movq %rbp, %rsp
@@ -94,13 +96,14 @@
;; movq %rsp, %rbp
;; unwind DefineNewFrame { offset_upward_to_caller_sp: 16, offset_downward_to_clobbers: 0 }
;; block0:
;; xorpd %xmm3, %xmm3, %xmm3
;; movdqa %xmm0, %xmm6
;; maxpd %xmm6, %xmm3, %xmm6
;; minpd %xmm6, const(0), %xmm6
;; roundpd $3, %xmm6, %xmm0
;; uninit %xmm4
;; xorpd %xmm4, %xmm4, %xmm4
;; movdqa %xmm0, %xmm8
;; maxpd %xmm8, %xmm4, %xmm8
;; minpd %xmm8, const(0), %xmm8
;; roundpd $3, %xmm8, %xmm0
;; addpd %xmm0, const(1), %xmm0
;; shufps $136, %xmm0, %xmm3, %xmm0
;; shufps $136, %xmm0, %xmm4, %xmm0
;; jmp label1
;; block1:
;; movq %rbp, %rsp