T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Re-implement atomic load and stores

Browse Source 
The AArch64 support was a bit broken and was using Armv7 style
barriers, which aren't required with Armv8 acquire-release
load/stores.

The fallback CAS loops and RMW, for AArch64, have also been updated
to use acquire-release, exclusive, instructions which, again, remove
the need for barriers. The CAS loop has also been further optimised
by using the extending form of the cmp instruction.

Copyright (c) 2021, Arm Limited.
This commit is contained in:
Sam Parker
2021-07-29 15:41:45 +01:00
parent 85f16f488d
commit cbb7229457
12 changed files with 564 additions and 220 deletions
Download Patch File Download Diff File Expand all files Collapse all files

117
cranelift/codegen/meta/src/shared/instructions.rs
View File

@@ -4600,8 +4600,7 @@ pub(crate) fn define(
r#"
Atomically load from memory at `p`.
This is a polymorphic instruction that can load any value type which has a memory
representation. It should only be used for integer types with 8, 16, 32 or 64 bits.
It should only be used for integer types with 32 or 64 bits.
This operation is sequentially consistent and creates happens-before edges that order
normal (non-atomic) loads and stores.
"#,
@@ -4613,14 +4612,124 @@ pub(crate) fn define(
.other_side_effects(true),
);
ig.push(
Inst::new(
"atomic_uload8",
r#"
Atomically load 8 bits from memory at `p` and zero-extend to either 32 or 64 bits.
This is equivalent to ``load.i8`` followed by ``uextend``.
This operation is sequentially consistent and creates happens-before edges that order
normal (non-atomic) loads and stores.
"#,
&formats.load_no_offset,
)
.operands_in(vec![MemFlags, p])
.operands_out(vec![a])
.can_load(true)
.other_side_effects(true),
);
ig.push(
Inst::new(
"atomic_uload16",
r#"
Atomically load 16 bits from memory at `p` and zero-extend to either 32 or 64 bits.
This is equivalent to ``load.i16`` followed by ``uextend``.
This operation is sequentially consistent and creates
happens-before edges that order normal (non-atomic) loads and stores.
"#,
&formats.load_no_offset,
)
.operands_in(vec![MemFlags, p])
.operands_out(vec![a])
.can_load(true)
.other_side_effects(true),
);
ig.push(
Inst::new(
"atomic_uload32",
r#"
Atomically load 32 bits from memory at `p` and zero-extend to 64 bits.
This is equivalent to ``load.i32`` followed by ``uextend``.
This operation is sequentially consistent and creates
happens-before edges that order normal (non-atomic) loads and stores.
"#,
&formats.load_no_offset,
)
.operands_in(vec![MemFlags, p])
.operands_out(vec![a])
.can_load(true)
.other_side_effects(true),
);
ig.push(
Inst::new(
"atomic_store",
r#"
Atomically store `x` to memory at `p`.
This is a polymorphic instruction that can store any value type with a memory
representation. It should only be used for integer types with 8, 16, 32 or 64 bits.
This is a polymorphic instruction that can store a 32 or 64-bit value.
This operation is sequentially consistent and creates happens-before edges that order
normal (non-atomic) loads and stores.
"#,
&formats.store_no_offset,
)
.operands_in(vec![MemFlags, x, p])
.can_store(true)
.other_side_effects(true),
);
ig.push(
Inst::new(
"atomic_store8",
r#"
Atomically store the low 8 bits of `x` to memory at `p`.
This is equivalent to ``ireduce.i8`` followed by ``store.i8``.
This operation is sequentially consistent and creates happens-before edges that order
normal (non-atomic) loads and stores.
"#,
&formats.store_no_offset,
)
.operands_in(vec![MemFlags, x, p])
.can_store(true)
.other_side_effects(true),
);
ig.push(
Inst::new(
"atomic_store16",
r#"
Atomically store the low 16 bits of `x` to memory at `p`.
This is equivalent to ``ireduce.i16`` followed by ``store.i16``.
This operation is sequentially consistent and creates happens-before edges that order
normal (non-atomic) loads and stores.
"#,
&formats.store_no_offset,
)
.operands_in(vec![MemFlags, x, p])
.can_store(true)
.other_side_effects(true),
);
ig.push(
Inst::new(
"atomic_store32",
r#"
Atomically store the low 32 bits of `x` to memory at `p`.
This is equivalent to ``ireduce.i32`` followed by ``store.i32``.
This operation is sequentially consistent and creates happens-before edges that order
normal (non-atomic) loads and stores.
"#,