stabilizer - grinnell collegestabilizer charlie curtsinger and emery d. berger statistically sound...
TRANSCRIPT
![Page 1: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/1.jpg)
STABILIZER
Charlie Curtsinger and Emery D. Berger
Statistically SoundPerformance Evaluation
![Page 2: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/2.jpg)
We all care about performance evaluation
STABILIZER
![Page 3: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/3.jpg)
We all care about performance evaluationWe’ve been doing it wrong
STABILIZER
![Page 4: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/4.jpg)
We all care about performance evaluationWe’ve been doing it wrong
STABILIZER
Repeated runs and error bars are not enough
![Page 5: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/5.jpg)
We all care about performance evaluationWe’ve been doing it wrong
STABILIZER
Repeated runs and error bars are not enough
We’re not measuring what we thought
![Page 6: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/6.jpg)
STABILIZERWe’ve been doing it wrong
![Page 7: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/7.jpg)
STABILIZER
Memory layout affects performance
We’ve been doing it wrong
![Page 8: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/8.jpg)
changing a program will change its layout
STABILIZER
Memory layout affects performance
We’ve been doing it wrong
![Page 9: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/9.jpg)
changing a program will change its layout
STABILIZER
Memory layout affects performance
no way to measure effect of change in isolation
We’ve been doing it wrong
![Page 10: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/10.jpg)
changing a program will change its layout
STABILIZER
Memory layout affects performance
STABILIZER eliminates the effect of layout
no way to measure effect of change in isolation
We’ve been doing it wrong
![Page 11: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/11.jpg)
changing a program will change its layout
STABILIZER
Memory layout affects performance
STABILIZER eliminates the effect of layout
no way to measure effect of change in isolation
enables sound performance evaluation
We’ve been doing it wrong
![Page 12: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/12.jpg)
changing a program will change its layout
STABILIZER
Memory layout affects performance
STABILIZER eliminates the effect of layout
no way to measure effect of change in isolation
Case Studies
enables sound performance evaluation
We’ve been doing it wrong
![Page 13: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/13.jpg)
changing a program will change its layout
STABILIZER
Memory layout affects performance
STABILIZER eliminates the effect of layout
no way to measure effect of change in isolation
evaluation of LLVM’s optimizations with STABILIZERCase Studies
enables sound performance evaluation
We’ve been doing it wrong
![Page 14: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/14.jpg)
STABILIZERWe’ve been doing it wrong
changing a program will change its layoutMemory layout affects performance
STABILIZER eliminates the effect of layout
no way to measure effect of change in isolation
evaluation of LLVM’s optimizations with STABILIZERCase Studies
enables sound performance evaluation
![Page 15: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/15.jpg)
A
Unsound performance evaluation
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
![Page 16: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/16.jpg)
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
A
Unsound performance evaluation
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
![Page 17: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/17.jpg)
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
A
Unsound performance evaluation
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; size_t meaning_of_life=42; for (size_t i = 0; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } for (size_t i = 16; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
![Page 18: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/18.jpg)
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
A
Unsound performance evaluation
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; size_t meaning_of_life=42; for (size_t i = 0; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } for (size_t i = 16; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
![Page 19: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/19.jpg)
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
A
Unsound performance evaluation
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; size_t meaning_of_life=42; for (size_t i = 0; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } for (size_t i = 16; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } asm("isync");}
![Page 20: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/20.jpg)
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
A
Unsound performance evaluation
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; size_t meaning_of_life=42; for (size_t i = 0; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } for (size_t i = 16; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } asm("isync");}
![Page 21: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/21.jpg)
A′int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
A
Unsound performance evaluation
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; size_t meaning_of_life=42; for (size_t i = 0; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } for (size_t i = 16; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } asm("isync");}
![Page 22: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/22.jpg)
A′A
Unsound performance evaluation
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; size_t meaning_of_life=42; for (size_t i = 0; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } for (size_t i = 16; i < size; i += 32) { asm("icbi 0,%0" : : "r"(p)); p += 32; } asm("isync");}
![Page 23: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/23.jpg)
0.00
0.25
0.50
0.75
1.00
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
A′A ×1 ×1
Which is faster?
![Page 24: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/24.jpg)
0.00
0.25
0.50
0.75
1.00
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
Is faster than ? A′ A
![Page 25: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/25.jpg)
0.00
0.25
0.50
0.75
1.00
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
Is faster than ? A′ A
![Page 26: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/26.jpg)
0.00
0.25
0.50
0.75
1.00
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
Is faster than ? A′ A
![Page 27: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/27.jpg)
0.00
0.25
0.50
0.75
1.00
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
Is faster than ? A′ A
![Page 28: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/28.jpg)
0.00
0.25
0.50
0.75
1.00
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
Is faster than ? A′ A
2.8% faster
![Page 29: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/29.jpg)
0.00
0.25
0.50
0.75
1.00
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
Is faster than ? A′ A
what about variance?
2.8% faster
![Page 30: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/30.jpg)
0
5
10
15
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
Which is faster?
A′A ×30 ×30
![Page 31: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/31.jpg)
0
5
10
15
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
Is faster than ? A′ A
![Page 32: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/32.jpg)
0
5
10
15
85.0 87.5 90.0 92.5 95.0Time (s)
Num
ber o
f run
s
VersionAA'
Is faster than ? A′ A
still2.8% faster
![Page 33: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/33.jpg)
Why is faster than ? A′ A
![Page 34: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/34.jpg)
Why is faster than ? A′ A
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
![Page 35: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/35.jpg)
Why is faster than ? A′ A
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
Was it the code change?
![Page 36: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/36.jpg)
Why is faster than ? A′ A
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
Was it the code change?
![Page 37: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/37.jpg)
Why is faster than ? A′ A
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
Or was it the new layout?
![Page 38: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/38.jpg)
Why is faster than ? A′ A
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
Or was it the new layout?
![Page 39: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/39.jpg)
Why is faster than ? A′ A
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
Mytkowicz et al. (ASPLOS’09)Layout biases measurement
![Page 40: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/40.jpg)
Layout biases measurementMytkowicz et al. (ASPLOS’09)
![Page 41: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/41.jpg)
Layout biases measurementMytkowicz et al. (ASPLOS’09)
Link OrderChanges function addresses
![Page 42: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/42.jpg)
Layout biases measurementMytkowicz et al. (ASPLOS’09)
Link Order
Environment Variable Size
Changes function addresses
Moves the program stack
![Page 43: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/43.jpg)
Layout biases measurementMytkowicz et al. (ASPLOS’09)
Link Order
Environment Variable Size
Changes function addresses
Moves the program stack
Larger than the impact of -O3
![Page 44: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/44.jpg)
Blame the cache
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
A
![Page 45: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/45.jpg)
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
map to same cache set
![Page 46: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/46.jpg)
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
conflict
map to same cache set
![Page 47: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/47.jpg)
A′
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
![Page 48: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/48.jpg)
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
A′
![Page 49: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/49.jpg)
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
map to same set
A′
![Page 50: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/50.jpg)
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
map to same set
A′
Nothing here
![Page 51: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/51.jpg)
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
map to same set
A′
Nothing here
no conflict
![Page 52: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/52.jpg)
A′
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
![Page 53: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/53.jpg)
A′
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
or branch predictor
![Page 54: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/54.jpg)
A′
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
or branch predictor
or TLB
![Page 55: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/55.jpg)
A′
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
or branch predictor
or TLB
or branch target predictor
![Page 56: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/56.jpg)
A′
Blame the cacheA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
or branch predictor
or TLB
or prefetcher
or branch target predictor
![Page 57: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/57.jpg)
Blame the hashA
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; }}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
int main(int argc, char **argv) { topFrame = (void**)__builtin_frame_address(0); setHandler(Trap::TrapSignal, onTrap); setHandler(SIGALRM, onTimer); setHandler(SIGSEGV, onFault); for(Function* f: functions) { f->setTrap(); } setTimer(interval); int r = stabilizer_main(argc, argv); return r;}
void setTimer(int msec) { struct itimerval timer;
timer.it_value.tv_sec = (msec - msec % 1000) / 1000; timer.it_value.tv_usec = 1000 * (msec % 1000); timer.it_interval.tv_sec = 0; timer.it_interval.tv_usec = 0; setitimer(ITIMER_REAL, &timer, 0);}
static void flush_icache(void* begin, size_t size) { uintptr_t p = (uintptr_t)begin & ~15UL; for (size_t i = 0; i < size; i += 16) { asm("icbi 0,%0" : : "r"(p)); p += 16; } asm("isync");}
DataHeapType* getDataHeap() { static char buf[sizeof(DataHeapType)]; static DataHeapType* _theDataHeap = new (buf) DataHeapType; return _theDataHeap;}
A′
![Page 58: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/58.jpg)
Is faster than ? A′ A
![Page 59: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/59.jpg)
it’s faster
Is faster than ? A′ A
Let’s do a poll
![Page 60: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/60.jpg)
it’s faster it’s faster
it’s faster
it’s faster
Is faster than ? A′ A
Let’s do a poll
![Page 61: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/61.jpg)
Do we trust this?
it’s faster it’s faster
it’s faster
it’s faster
![Page 62: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/62.jpg)
But it’s faster
Is faster than ? A′ A
![Page 63: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/63.jpg)
it’s faster it’s slower they’re the same
But it’s faster
Is faster than ? A′ A
![Page 64: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/64.jpg)
it’s faster it’s slower they’re the same
But it ran faster!
![Page 65: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/65.jpg)
it’s faster it’s slower they’re the same
But it ran faster!What if we only talk to Bob?
![Page 66: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/66.jpg)
it’s faster
But it ran faster!What if we only use this layout?
it’s slower they’re the same
![Page 67: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/67.jpg)
it’s faster
But it ran faster!What if we only use this layout?
![Page 68: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/68.jpg)
it’s faster
But it ran faster!What if we only use this layout?
Upgrade libc
![Page 69: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/69.jpg)
it’s faster
But it ran faster!What if we only use this layout?
Upgrade libcChanges layout
![Page 70: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/70.jpg)
it’s faster
But it ran faster!What if we only use this layout?
Change Username
![Page 71: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/71.jpg)
it’s faster
But it ran faster!What if we only use this layout?
Change UsernameChanges layout
![Page 72: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/72.jpg)
Layout is Brittle
it’s faster
But it ran faster!What if we only use this layout?
Run in a new directory
![Page 73: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/73.jpg)
Layout is Brittle
it’s faster
But it ran faster!What if we only use this layout?
Run in a new directory
Changes layout
![Page 74: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/74.jpg)
Layout is Brittle
But it ran faster!What if we only use this layout?
![Page 75: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/75.jpg)
Layout is Brittle
But it ran faster!What if we only use this layout?
Layout biases measurementMytkowicz et al. (ASPLOS’09)
![Page 76: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/76.jpg)
Layout is Brittle
But it ran faster!What if we only use this layout?
Layout biases measurementMytkowicz et al. (ASPLOS’09)
Can we eliminate the effect of layout?
![Page 77: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/77.jpg)
But it ran faster!What if we only use this layout?
Layout biases measurement
Can we eliminate the effect of layout?
YES
![Page 78: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/78.jpg)
STABILIZERMemory layout affects performance
STABILIZER eliminates the effect of layoutenables sound performance evaluation
evaluation of LLVM’s optimizations with STABILIZERCase Studies
makes performance evaluation difficult
![Page 79: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/79.jpg)
STABILIZERMemory layout affects performance
STABILIZER eliminates the effect of layoutenables sound performance evaluation
evaluation of LLVM’s optimizations with STABILIZERCase Studies
makes performance evaluation difficult
![Page 80: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/80.jpg)
Layout biases measurement
STABILIZERrandomizes layout
![Page 81: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/81.jpg)
Layout biases measurement
STABILIZER
function addresses
randomizes layout
![Page 82: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/82.jpg)
Layout biases measurement
STABILIZER
function addresses stack frame sizes
randomizes layout
![Page 83: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/83.jpg)
Layout biases measurement
STABILIZER
function addresses stack frame sizesheap allocations
randomizes layout
![Page 84: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/84.jpg)
STABILIZERrandomizes layout
function addresses stack frame sizesheap allocations
repeatedly
Layout biases measurement
![Page 85: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/85.jpg)
STABILIZERrandomizes layout
function addresses stack frame sizesheap allocations
repeatedly
Layout biases measurement
during execution
![Page 86: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/86.jpg)
STABILIZERrandomizes layout
function addresses stack frame sizesheap allocations
repeatedly
Layout biases measurement
![Page 87: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/87.jpg)
STABILIZERrandomizes layout
function addresses stack frame sizesheap allocations
repeatedly
Layout biases measurementa completely random layout
cannot bias results
![Page 88: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/88.jpg)
A′A ×30 ×30
Sound Performance Evaluation
![Page 89: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/89.jpg)
A ×30 ×30
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A′
Sound Performance Evaluation
![Page 90: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/90.jpg)
Is faster than ? A′ A
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
![Page 91: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/91.jpg)
Is faster than ?A
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A′
![Page 92: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/92.jpg)
Is faster than ?A
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A′
![Page 93: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/93.jpg)
Is faster than ?A
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A′
![Page 94: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/94.jpg)
Is faster than ?A
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A′
![Page 95: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/95.jpg)
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A A′
The Statistical Approach
If =
![Page 96: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/96.jpg)
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A A′
The Statistical Approach
If =
hypothesis testing
![Page 97: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/97.jpg)
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A A′what is the probability of measuring
a speedup this large by chance?
The Statistical Approach
If =
hypothesis testing
![Page 98: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/98.jpg)
what is the probability of measuring a speedup this large by chance?
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
easy to compute for the normal distribution
A A′If =
![Page 99: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/99.jpg)
easy to compute for the normal distribution
A A′If =
![Page 100: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/100.jpg)
STABILIZERrandomizes layoutrepeatedly
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
what is the probability of measuring a speedup this large by chance?
![Page 101: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/101.jpg)
STABILIZERrandomizes layoutrepeatedly
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
what is the probability of measuring a speedup this large by chance?
if there is alow probability
![Page 102: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/102.jpg)
STABILIZERrandomizes layoutrepeatedly
this speedup is real
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
![Page 103: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/103.jpg)
STABILIZERrandomizes layoutrepeatedly
this speedup is real
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
not due to the effect on memory layout
![Page 104: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/104.jpg)
this speedup is real
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Sound Performance EvaluationSTABILIZER
randomizes layoutrepeatedly
what does re-randomization do?
not due to the effect on memory layout
![Page 105: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/105.jpg)
0.0
0.2
0.4
350 360 370 380 390 400Time (s)
Prob
abili
ty D
ensi
tySTABILIZER
randomizes layoutrepeatedly
![Page 106: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/106.jpg)
0.0
0.2
0.4
350 360 370 380 390 400Time (s)
Prob
abili
ty D
ensi
tySTABILIZER
randomizes layoutrepeatedly
one random layout per-run
![Page 107: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/107.jpg)
STABILIZERrandomizes layoutrepeatedly
0.0
0.2
0.4
350 360 370 380 390 400Time (s)
Prob
abili
ty D
ensi
ty
one random layout per-run
![Page 108: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/108.jpg)
STABILIZERrandomizes layoutrepeatedly
0.0
0.2
0.4
350 360 370 380 390 400Time (s)
Prob
abili
ty D
ensi
ty
many random layouts in each run
one random layout per-run
![Page 109: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/109.jpg)
STABILIZERrandomizes layoutrepeatedly
![Page 110: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/110.jpg)
STABILIZER generates a new random layout every ½ second
STABILIZERrandomizes layoutrepeatedly
![Page 111: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/111.jpg)
STABILIZER generates a new random layout every ½ second
Total execution time isthe sum of all periods
STABILIZERrandomizes layoutrepeatedly
![Page 112: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/112.jpg)
STABILIZER generates a new random layout every ½ second
Total execution time isthe sum of all periods
The sum of a sufficient number of independent, identically distributed random
variables is approximately normally distributed.
STABILIZERrandomizes layoutrepeatedly
![Page 113: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/113.jpg)
STABILIZER generates a new random layout every ½ second
Total execution time isthe sum of all periods
The sum of a sufficient number of independent, identically distributed random
variables is approximately normally distributed.
STABILIZERrandomizes layoutrepeatedly
![Page 114: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/114.jpg)
STABILIZER generates a new random layout every ½ second
Total execution time isthe sum of all periods
The sum of a sufficient number of independent, identically distributed random
variables is approximately normally distributed.
STABILIZERrandomizes layoutrepeatedly
![Page 115: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/115.jpg)
Central Limit Theorem
The sum of a sufficient number of independent, identically distributed random
variables is approximately normally distributed.
![Page 116: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/116.jpg)
Central Limit Theoremexecution times are normally distributed
The sum of a sufficient number of independent, identically distributed random
variables is approximately normally distributed.
![Page 117: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/117.jpg)
STABILIZERMemory layout affects performance
STABILIZER eliminates the effect of layoutenables sound performance evaluation
evaluation of LLVM’s optimizations with STABILIZERCase Studies
makes performance evaluation difficult
![Page 118: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/118.jpg)
STABILIZER
makes performance evaluation difficultMemory layout affects performance
STABILIZER eliminates the effect of layoutenables sound performance evaluation
evaluation of LLVM’s optimizations with STABILIZERCase Studies
![Page 119: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/119.jpg)
Case Studies
evaluation of LLVM’s optimizations with STABILIZER
![Page 120: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/120.jpg)
Case Studies
on each benchmark
evaluation of LLVM’s optimizations with STABILIZER
![Page 121: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/121.jpg)
Case Studies
on each benchmark
across the whole benchmark suite
evaluation of LLVM’s optimizations with STABILIZER
![Page 122: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/122.jpg)
Case Studies
on each benchmark
across the whole benchmark suite
evaluation of LLVM’s optimizations with STABILIZER
first, build benchmarks with STABILIZER
![Page 123: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/123.jpg)
Build programs with STABILIZER
![Page 124: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/124.jpg)
Build programs with STABILIZER
> szc main.c
![Page 125: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/125.jpg)
> szc main.c
Build programs with STABILIZER
![Page 126: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/126.jpg)
> szc main.c-Rcode
Build programs with STABILIZER
![Page 127: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/127.jpg)
> szc main.c-Rcode -Rheap -Rstack
Build programs with STABILIZER
![Page 128: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/128.jpg)
> szc main.c-Rcode -Rheap -Rstack
Build programs with STABILIZER
now run the benchmarks
![Page 129: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/129.jpg)
Run benchmarks as usual
A′A ×30 ×30
![Page 130: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/130.jpg)
Run benchmarks as usual
A′A ×30 ×30
drop the results into R
![Page 131: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/131.jpg)
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Run benchmarks as usual
A′A ×30 ×30
drop the results into R
![Page 132: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/132.jpg)
Is faster than ?A
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A′
![Page 133: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/133.jpg)
Is faster than ?A
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
A′
![Page 134: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/134.jpg)
A′
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
If = A
![Page 135: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/135.jpg)
If = A′
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Percen
t of Ob
served
Runtim
es
A
![Page 136: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/136.jpg)
A′If = A
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Percen
t of Ob
served
Runtim
es
what is the probability of measuring a difference at least this large?
![Page 137: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/137.jpg)
what is the probability of measuring a difference at least this large?
A′
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Percen
t of Ob
served
Runtim
es
The Student’s t-test
If = A
![Page 138: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/138.jpg)
what is the probability of measuring a difference at least this large?
A′
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Percen
t of Ob
served
Runtim
es
The Student’s t-test
If = A
t.test(times. A′, times.A)
![Page 139: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/139.jpg)
what is the probability of measuring a difference at least this large?
A′If = A
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Percen
t of Ob
served
Runtim
es
The Student’s t-testIf p-value
![Page 140: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/140.jpg)
A′
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Percen
t of Ob
served
Runtim
es
The Student’s t-testIf p-value ≤ 5%
If = A
what is the probability of measuring a difference at least this large?
![Page 141: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/141.jpg)
A′
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Percen
t of Ob
served
Runtim
es
The Student’s t-testIf p-value ≤ 5%
If = A
95% Confidence
what is the probability of measuring a difference at least this large?
![Page 142: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/142.jpg)
If = A′
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Percen
t of Ob
served
Runtim
es
A
The Student’s t-testIf p-value ≤ 5%
we reject the null hypothesis
what is the probability of measuring a difference at least this large?
![Page 143: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/143.jpg)
≠ A
The Student’s t-testIf p-value ≤ 5%
we reject the null hypothesis
0%
10%
20%
30%
40%
85.0 87.5 90.0 92.5 95.0Time (s)
Percen
t of Ob
served
Runtim
es
Random chance not responsible for the measured difference
A′
![Page 144: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/144.jpg)
≠ A
The difference is real
A′
![Page 145: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/145.jpg)
−10%
0%
10%
20%
libquantum
milc
bzip2
sphinx3
namd lbm
perlbench
hmmer
h264ref
cactusADM wrf
sjeng
gobmk
gromacs
zeusmp mcf
gcc
astar
Speedup Significant
Yes
No
Speedup of -O2 over -O1
![Page 146: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/146.jpg)
−10%
0%
10%
20%
libquantum
milc
bzip2
sphinx3
namd lbm
perlbench
hmmer
h264ref
cactusADM wrf
sjeng
gobmk
gromacs
zeusmp mcf
gcc
astar
Speedup Significant
Yes
No
Speedup of -O2 over -O1
![Page 147: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/147.jpg)
−10%
0%
10%
20%
libquantum
milc
bzip2
sphinx3
namd lbm
perlbench
hmmer
h264ref
cactusADM wrf
sjeng
gobmk
gromacs
zeusmp mcf
gcc
astar
Speedup Significant
Yes
No
Speedup of -O2 over -O1
![Page 148: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/148.jpg)
−10%
0%
10%
20%
libquantum
milc
bzip2
sphinx3
namd lbm
perlbench
hmmer
h264ref
cactusADM wrf
sjeng
gobmk
gromacs
zeusmp mcf
gcc
astar
Speedup Significant
Yes
No
Speedup of -O2 over -O1
![Page 149: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/149.jpg)
−10%
0%
10%
20%
bzip2
gobmk
zeusmp
libquantum wrf
astar mcf
hmmer
milc
namd gcc
lbmgromacs
h264ref
cactusADM
perlbench
sphinx3
sjeng
Speedup Significant
Yes
No
Speedup of -O3 over -O2
![Page 150: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/150.jpg)
Speedup of -O3 over -O2
0.0%
0.5%
1.0%
1.5%
bzip2
gobmk
zeusmp
libquantum wrf
astar mcf
hmmer
milc
namd gcc
lbmgromacs
h264ref
cactusADM
perlbench
sphinx3
sjeng
Speedup Significant
Yes
No
![Page 151: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/151.jpg)
0.0%
0.5%
1.0%
1.5%
bzip2
gobmk
zeusmp
libquantum wrf
astar mcf
hmmer
milc
namd gcc
lbmgromacs
h264ref
cactusADM
perlbench
sphinx3
sjeng
Speedup Significant
Yes
No
Speedup of -O3 over -O2
![Page 152: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/152.jpg)
0.0%
0.5%
1.0%
1.5%
bzip2
gobmk
zeusmp
libquantum wrf
astar mcf
hmmer
milc
namd gcc
lbmgromacs
h264ref
cactusADM
perlbench
sphinx3
sjeng
Speedup Significant
Yes
No
Speedup of -O3 over -O2
![Page 153: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/153.jpg)
0.0%
0.5%
1.0%
1.5%
bzip2
gobmk
zeusmp
libquantum wrf
astar mcf
hmmer
milc
namd gcc
lbmgromacs
h264ref
cactusADM
perlbench
sphinx3
sjeng
Speedup Significant
Yes
No
Speedup of -O3 over -O2
![Page 154: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/154.jpg)
0.0%
0.5%
1.0%
1.5%
bzip2
gobmk
zeusmp
libquantum wrf
astar mcf
hmmer
milc
namd gcc
lbmgromacs
h264ref
cactusADM
perlbench
sphinx3
sjeng
Speedup Significant
Yes
No
What do the results mean?
![Page 155: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/155.jpg)
Comparing optimizations
-O2 -O3×30 ×30
![Page 156: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/156.jpg)
-O2 -O3
×30 ×30lbm lbm
×30 ×30
Comparing optimizations
![Page 157: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/157.jpg)
-O2 -O3
×30 ×30lbm lbm
×30 ×30astar astar
×30 ×30
Comparing optimizations
![Page 158: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/158.jpg)
-O2 -O3
×30 ×30lbm lbm
×30 ×30astar astar
...
×30 ×30
Comparing optimizations
![Page 159: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/159.jpg)
-O2 -O3×30 ×30
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
Comparing optimizations
![Page 160: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/160.jpg)
-O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
Is faster than ?
![Page 161: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/161.jpg)
-O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
Is faster than ?
![Page 162: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/162.jpg)
-O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
Is faster than ?
![Page 163: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/163.jpg)
-O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
Is faster than ?
![Page 164: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/164.jpg)
If = -O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
![Page 165: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/165.jpg)
If = -O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
![Page 166: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/166.jpg)
If = -O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
what is the probability of measuring these differences?
![Page 167: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/167.jpg)
If = -O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
what is the probability of measuring these differences?
Analysis of Variance
![Page 168: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/168.jpg)
If = -O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
what is the probability of measuring these differences?
Analysis of Varianceaov(time~opt+Error(benchmark/opt), times)
![Page 169: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/169.jpg)
If = -O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
what is the probability of measuring these differences?
Analysis of VarianceIf p-value
![Page 170: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/170.jpg)
If = -O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
what is the probability of measuring these differences?
Analysis of VarianceIf p-value ≤ 5%
![Page 171: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/171.jpg)
If = -O2-O3
0%
1%
2%
3%
4%
0 25 50 75 100Time (s)
Perc
ent o
f Obs
erve
d Ru
ntim
es
Version−O2−O3
what is the probability of measuring these differences?
Analysis of VarianceIf p-value ≤ 5%
we reject the null hypothesis
![Page 172: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/172.jpg)
Analysis of VarianceIf p-value ≤ 5%
we reject the null hypothesis
-O3 -O2vs
![Page 173: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/173.jpg)
Analysis of VarianceIf p-value ≤ 5%
we reject the null hypothesis
p-value = 26.4%
-O3 -O2vs
![Page 174: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/174.jpg)
Analysis of VarianceIf p-value ≤ 5%
we reject the null hypothesis
p-value = 26.4%
-O3 -O2vs
are we 73.6% confident?
![Page 175: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/175.jpg)
Analysis of VarianceIf p-value ≤ 5%
we reject the null hypothesis
p-value = 26.4%
-O3 -O2vs
are we 73.6% confident?one in four experiments will show an
effect that does not exist!
![Page 176: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/176.jpg)
Analysis of VarianceIf p-value ≤ 5%
we reject the null hypothesis
p-value = 26.4%fail to reject the null hypothesis
-O3 -O2vs
![Page 177: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/177.jpg)
-O3
Analysis of Variance
we reject the null hypothesis
-O2The effect of over is indistinguishable from noise
If p-value ≤ 5%
![Page 178: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/178.jpg)
-O3
Analysis of Variance
we reject the null hypothesis
-O2The effect of over is indistinguishable from noise
If p-value ≤ 5%
Did STABILIZER hide the effect?
![Page 179: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/179.jpg)
Runtime with -O2
Runtime with -O3
Runtime with -O1
Runtime with -O0
Execution Time
Did STABILIZER hide the effect?
![Page 180: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/180.jpg)
STABILIZ
ER
STABILIZ
ER
STABILIZ
ERSTA
BILIZER
Execution Time
Runtime with -O2
Runtime with -O3
Runtime with -O1
Runtime with -O0
Did STABILIZER hide the effect?
![Page 181: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/181.jpg)
STABILIZ
ER
STABILIZ
ER
STABILIZ
ERSTA
BILIZER
Execution Time
Runtime with -O2
Runtime with -O3
Runtime with -O1
Runtime with -O0
speedups
Did STABILIZER hide the effect?
![Page 182: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/182.jpg)
Runtime with -O2
Runtime with -O3
Runtime with -O1
Runtime with -O0
STABILIZ
ER
STABILIZ
ER
STABILIZ
ERSTA
BILIZER
Execution Time
speedups
Did STABILIZER hide the effect?
![Page 183: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/183.jpg)
SPECint SPECfp Summary
0%
10%
20%
30%
40%
perlbe
nch gcc
gobm
kh2
64ref
sjeng
astar
bzip2
libqu
antu
mhm
mer mcfca
ctusA
DMze
usmp
wrfgr
omac
ssp
hinx3 lbm milc
namd
hmea
nmed
ian
% In
crea
se in
Exe
cutio
n Ti
me
STABILIZER is fast enough
![Page 184: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/184.jpg)
STABILIZERMemory layout affects performance
STABILIZER eliminates the effect of layout
showed that -O3 does not have a statistically significant effect across our benchmarks
Case Studies
random layout enables sound performance evaluation
makes performance evaluation difficult
![Page 185: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/185.jpg)
STABILIZERMemory layout affects performance
STABILIZER eliminates the effect of layout
showed that -O3 does not have a statistically significant effect across our benchmarks
Case Studies
random layout enables sound performance evaluation
makes performance evaluation difficult
![Page 186: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/186.jpg)
STABILIZERMemory layout affects performance
STABILIZER eliminates the effect of layout
showed that -O3 does not have a statistically significant effect across our benchmarks
Case Studies
random layout enables sound performance evaluation
makes performance evaluation difficult
![Page 187: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/187.jpg)
STABILIZERMemory layout affects performance
STABILIZER eliminates the effect of layout
showed that -O3 does not have a statistically significant effect across our benchmarks
Case Studies
random layout enables sound performance evaluation
makes performance evaluation difficult
![Page 188: STABILIZER - Grinnell CollegeSTABILIZER Charlie Curtsinger and Emery D. Berger Statistically Sound Performance Evaluation](https://reader034.vdocuments.mx/reader034/viewer/2022042803/5f4802c7edddab4d4960aca3/html5/thumbnails/188.jpg)
available at
stabilizer-tool.org
STABILIZER