T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Patch qemu in CI to fix madvise semantics. (#3770)

Browse Source 
We currently skip some tests when running our qemu-based tests for
aarch64 and s390x. Qemu has broken madvise(MADV_DONTNEED) semantics --
specifically, it just ignores madvise() [1].

We could continue to whack-a-mole the tests whenever we create new
functionality that relies on madvise() semantics, but ideally we'd just
have emulation that properly emulates!

The earlier discussions on the qemu mailing list [2] had a proposed
patch for this, but (i) this patch doesn't seem to apply cleanly anymore
(it's 3.5 years old) and (ii) it's pretty complex due to the need to
handle qemu's ability to emulate differing page sizes on host and guest.

It turns out that we only really need this for CI when host and guest
have the same page size (4KiB), so we *could* just pass the madvise()s
through. I wouldn't expect such a patch to ever land upstream in qemu,
but it satisfies our needs I think. So this PR modifies our CI setup to
patch qemu before building it locally with a little one-off patch.

[1]
https://github.com/bytecodealliance/wasmtime/pull/2518#issuecomment-747280133

[2]
https://lists.gnu.org/archive/html/qemu-devel/2018-08/msg05416.html
This commit is contained in:
Chris Fallin
2022-02-07 15:56:54 -08:00
committed by GitHub
parent 43b37944ff
commit ddd39cdb84
4 changed files with 63 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
crates/runtime/src/instance/allocator/pooling.rs
View File

@@ -1748,11 +1748,6 @@ mod test {
#[cfg(all(unix, target_pointer_width = "64", feature = "async"))]
#[test]
fn test_stack_zeroed() -> Result<()> {
// https://github.com/bytecodealliance/wasmtime/pull/2518#issuecomment-747280133
if std::env::var("WASMTIME_TEST_NO_HOG_MEMORY").is_ok() {
return Ok(());
}
let allocator = PoolingInstanceAllocator::new(
PoolingAllocationStrategy::NextAvailable,
ModuleLimits {

23
crates/runtime/src/memfd.rs
View File

@@ -554,10 +554,6 @@ mod test {
#[test]
fn instantiate_no_image() {
if skip_tests_due_to_qemu_madvise_semantics() {
return;
}
// 4 MiB mmap'd area, not accessible
let mut mmap = Mmap::accessible_reserved(0, 4 << 20).unwrap();
// Create a MemFdSlot on top of it
@@ -590,10 +586,6 @@ mod test {
#[test]
fn instantiate_image() {
if skip_tests_due_to_qemu_madvise_semantics() {
return;
}
// 4 MiB mmap'd area, not accessible
let mut mmap = Mmap::accessible_reserved(0, 4 << 20).unwrap();
// Create a MemFdSlot on top of it
@@ -637,19 +629,4 @@ mod test {
let slice = mmap.as_slice();
assert_eq!(&[1, 2, 3, 4], &slice[4096..4100]);
}
/// qemu's madvise implementation does not implement the
/// "flash-reset back to zero or CoW backing" semantics that Linux
/// does. Our CI setup uses qemu (in usermode-binary mode, not
/// whole-system mode) to run tests on aarch64 and s390x. We want
/// to skip these tests when under qemu, but not when someone is
/// developing natively on one of these architectures. So instead,
/// we dynamically detect an environment variable that our CI
/// setup sets.
///
/// See `skip_pooling_allocator_tests()` in `tests/all/main.rs`
/// for more.
fn skip_tests_due_to_qemu_madvise_semantics() -> bool {
std::env::var("WASMTIME_TEST_NO_HOG_MEMORY").is_ok()
}
}