T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Be more forgiving about what's a "slow" test.

Browse Source 
This was supposed to be Q3 + 1.5 IQR, but a braino meant we actually used
Q3 + 2/3 IQR.

Since the distribution of test case times is far from gaussian, bump the
"slow" limit up even further to Q3 + 3 IQR.
This commit is contained in:
Jakob Stoklund Olesen
2017-11-22 11:48:47 -08:00
parent 8e2ce6ded2
commit c810b21488
1 changed files with 6 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
cranelift/src/filetest/runner.rs
View File

@@ -302,9 +302,12 @@ impl TestRunner {
// Inter-quartile range. // Inter-quartile range.
let iqr = q3 - q1; let iqr = q3 - q1;
// Cut-off for what we consider a 'slow' test: 1.5 IQR from the 75% quartile.
// These are the data points that would be plotted as outliers outside a box plot. // Cut-off for what we consider a 'slow' test: 3 IQR from the 75% quartile.
let cut = q3 + iqr * 2 / 3; //
// Q3 + 1.5 IQR are the data points that would be plotted as outliers outside a box plot,
// but we have a wider distribution of test times, so double it to 3 IQR.
let cut = q3 + iqr * 3;
if cut > *times.last().unwrap() { if cut > *times.last().unwrap() {
return; return;
} }