COSCUP 2014 : open source compiler

Download COSCUP 2014 : open source compiler

Post on 19-Aug-2014

2.454 views

Category:

Engineering

16 download

DESCRIPTION

COSCUP 2014 : open source compiler

TRANSCRIPT

Open Source Compiler COSCUP'2014 Date : July 19th, 2014 Kito Cheng kito.cheng@gmail.com 2 Compiler Team 3 Open Source Compiler 4 yum update -ysudo apt-get upgrade Compiler? 5 6 7 8 Compiler 9 Compiler Debug! 10 GNU ld.bfd / ld.gold gdb as/objdump libstdc++ libgcc 11 GNU LLVM ld.bfd / ld.gold lld / mclinker gdb lldb as/objdump MC layer in LLVM libstdc++ libc++ libgcc libcompiler-rt 12 binutils vs MC Layer binutils Library , . LLVM MC Layer , assembler disassembler Library . Toolchain 13 libstdc++ vs libc++ Linux C++ libstdc++ 14 libstdc++ vs libc++ Linux C++ libstdc++ std::string... 15 libstdc++ vs libc++ Linux C++ libstdc++ std::string... libc++ ! 16 libstdc++ vs libc++ Linux C++ libstdc++ std::string... libc++ ! STLPort : 17 libstdc++ vs libc++ Linux C++ libstdc++ std::string... libc++ ! STLPort : STLPort C++ Runtime Library 18 VM/JIT New Programming Language / 19 VM/JIT 20 VM/JIT 21 VM/JIT Kaffe VM : JIT Interpreter 22 VM/JIT Kaffe VM : JIT Interpreter Just in time 23 VM/JIT Kaffe VM : JIT Interpreter Just in time Just too late! 24 Pyston FTLWebKits LLVM based JIT 25 New Programming Language 26 New Programming Language Native Execution ... 27 New Programming Language Native Execution ... code gen 28 New Programming Language Native Execution ... code gen C code Compiler 29 New Programming Language Native Execution ... code gen C code Compiler Compiler 30 New Programming Language GCC ... 31 New Programming Language GCC ... , GCC IR - GIMPLE XD : GPLv3 32 New Programming Language GCC ... , GCC IR - GIMPLE XD : GPLv3 LLVM ! 33 Rust 34 / youcompleteme clang static analyzer 35 in Compiler Address-sanitizer Undefined-sanitizer Thread-sanitizer 36 Address-sanitizer :) Valgrind ! 37 int main(int argc, char **argv) { int stack_array[100]; stack_array[1] = 0; return stack_array[argc + 100]; // BOOM } ================================================================= ==28706==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fff61e1f344 at pc 0x4a5dfb bp 0x7fff61e1f170 sp 0 READ of size 4 at 0x7fff61e1f344 thread T0 #0 0x4a5dfa in main /home/kito/test.cpp:4 #1 0x7ff11a8a1d64 in __libc_start_main (/lib64/libc.so.6+0x21d64) #2 0x404c98 (/home/kito/a.out+0x404c98) Address 0x7fff61e1f344 is located in stack of thread T0 at offset 436 in frame #0 0x4a5d29 in main /home/kito/test.cpp:1 This frame has 1 object(s): [32, 432) 'stack_array' 0x10006c3bbe60: 00 00 00 00 00 00 00 00[f4]f4 f3 f3 f3 f3 00 00 ... 0x10006c3bbeb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 ... 38 ==12254==ERROR: AddressSanitizer: heap-use-after-free on address 0x60200000eff0 at pc 0x4a5db4 bp 0x7fff3ff57520 sp 0x7fff3ff57518 READ of size 4 at 0x60200000eff0 thread T0 #0 0x4a5db3 in main /home/kito/coscup2014/use-after-free.c:6 #1 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64) #2 0x404c98 (/home/kito/coscup2014/a.out+0x404c98) 0x60200000eff0 is located 0 bytes inside of 4-byte region [0x60200000eff0,0x60200000eff4) freed by thread T0 here: #0 0x476c79 in __interceptor_free /home/kito/gcc/gcc-src/libsanitizer/asan/asan_malloc_linux.cc:63 #1 0x4a5d7c in main /home/kito/coscup2014/use-after-free.c:5 #2 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64) previously allocated by thread T0 here: #0 0x476f19 in __interceptor_malloc /home/kito/gcc/gcc-src/libsanitizer/asan/asan_malloc_linux.cc:73 #1 0x4a5d2b in main /home/kito/coscup2014/use-after-free.c:3 #2 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64) #include int main() { int *a = malloc(sizeof(int)); *a = 100; free(a); return *a; } 39 free/delete/delete[] int main() { int *arr = new int[10]; delete arr; return 0; } ================================================================= ==12421==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new [] vs operator delete) on 0x60400000dfd0 #0 0x478219 in operator delete(void*) /home/kito/gcc/gcc-src/libsanitizer/asan/asan_new_delete.cc:85 #1 0x4a5efb in main /home/kito/coscup2014/mismatch-delete.cpp:3 #2 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64) #3 0x404e58 (/home/kito/coscup2014/a.out+0x404e58) 0x60400000dfd0 is located 0 bytes inside of 40-byte region [0x60400000dfd0,0x60400000dff8) allocated by thread T0 here: #0 0x477e29 in operator new[](unsigned long) /home/kito/gcc/gcc-src/libsanitizer/asan/asan_new_delete.cc:55 #1 0x4a5eeb in main /home/kito/coscup2014/mismatch-delete.cpp:2 #2 0x3c52221d64 in __libc_start_main (/lib64/libc.so.6+0x3c52221d64) 40 Address-sanitizer Valgrind Address Sanitizer Heap Y Y Stack N Y N Y (free/delete) Y Y ( : ) N Y( ) Y N free/delete/delete[] Y Y 10x-30x 1.5x-3x 41 Undefined-Sanitizer Undefined behavior 42 Undefined-Sanitizer Undefined behavior k standard undefined behavior... 43 Undefined-Sanitizer Undefined behavior k standard undefined behavior... c99 UB 1x 44 Undefined-Sanitizer Undefined behavior k standard undefined behavior... c99 UB 1x Undefined behavior !!!!! 45 Undefined-Sanitizer Undefined behavior k standard undefined behavior... c99 UB 1x Undefined behavior !!!!! UB...XD 46 Divde by 0 int main(int argc, const char *argv[]){ return argc/0; } div0.cpp:2:14: runtime error: division by zero Floating point exception 47 Deference Null pointer int main(int argc, const char *argv[]){ int *a = nullptr; return *a; } derefnull.cpp:3:11: runtime error: load of null pointer of type 'int' Segmentation fault 48 Shift int main(int argc, const char *argv[]){ return argc >> 32; } shift.cpp:2:15: runtime error: shift exponent 32 is too large for 32-bit type 'int' 49 Signed Integer Overflow #include int main(int argc, const char *argv[]){ int a = INT_MAX; return a + argc; } overflow.cpp:4:14: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int' 50 Thread-Sanitizer Race Condition ! Race Condition , Thread-Sanitizer . 51 Race Condition #include #include int Global; void *Thread1(void *x) { Global++; return NULL; } void *Thread2(void *x) { Global--; return NULL; } int main() { pthread_t t[2]; pthread_create(&t[0], NULL, Thread1, NULL); pthread_create(&t[1], NULL, Thread2, NULL); pthread_join(t[0], NULL); pthread_join(t[1], NULL); } 52 Race Condition #include #include int Global; void *Thread1(void *x) { Global++; return NULL; } void *Thread2(void *x) { Global--; return NULL; } int main() { pthread_t t[2]; pthread_create(&t[0], NULL, Thread1, NULL); pthread_create(&t[1], NULL, Thread2, NULL); pthread_join(t[0], NULL); pthread_join(t[1], NULL); } ================== WARNING: ThreadSanitizer: data race (pid=21757) Write of size 4 at 0x7ffa3e002ef4 by thread T2: #0 Thread2 /home/kito/coscup2014/race.c:12 (race+0x0000000c1a75) Previous write of size 4 at 0x7ffa3e002ef4 by thread T1: #0 Thread1 /home/kito/coscup2014/race.c:7 (race+0x0000000c1a05) Location is global 'Global' of size 4 at 0x7ffa3e002ef4 (race+0x000000e03ef4) Thread T2 (tid=21760, running) created by main thread at: #0 pthread_create /home/kito/llvm/src/projects/compiler- rt/lib/tsan/rtl/tsan_interceptors.cc:842 (race+0x00000005e1f1) #1 main /home/kito/coscup2014/race.c:19 (race+0x0000000c1b03) Thread T1 (tid=21759, finished) created by main thread at: #0 pthread_create /home/kito/llvm/src/projects/compiler- rt/lib/tsan/rtl/tsan_interceptors.cc:842 (race+0x00000005e1f1) #1 main /home/kito/coscup2014/race.c:18 (race+0x0000000c1ad9) SUMMARY: ThreadSanitizer: data race /home/kito/coscup2014/race.c:12 Thread2 ================== ThreadSanitizer: reported 1 warnings 53 Race Condition #include #include int Global; pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; void *Thread1(void *x) { pthread_mutex_lock(&lock); Global++; pthread_mutex_unlock(&lock); return NULL; } void *Thread2(void *x) { Global--; return NULL; } int main() { pthread_t t[2]; pthread_create(&t[0], NULL, Thread1, NULL); pthread_create(&t[1], NULL, Thread2, NULL); pthread_join(t[0], NULL); pthread_join(t[1], NULL); } 54 Race Condition #include #include int Global; pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; void *Thread1(void *x) { pthread_mutex_lock(&lock); Global++; pthread_mutex_unlock(&lock); return NULL; } void *Thread2(void *x) { Global--; return NULL; } int main() { pthread_t t[2]; pthread_create(&t[0], NULL, Thread1, NULL); pthread_create(&t[1], NULL, Thread2, NULL); pthread_join(t[0], NULL); pthread_join(t[1], NULL); } ================== WARNING: ThreadSanitizer: data race (pid=21765) Write of size 4 at 0x7feaa10dcf20 by thread T2: #0 Thread2 /home/kito/coscup2014/race-2.c:15 (race- 2+0x0000000c1ad5) Previous write of size 4 at 0x7feaa10dcf20 by thread T1 (mutexes: write M0): #0 Thread1 /home/kito/coscup2014/race-2.c:9 (race- 2+0x0000000c1a57) Location is global 'Global' of size 4 at 0x7feaa10dcf20 (race- 2+0x000000e03f20) Mutex M0 (0x7feaa10dcef8) created at: #0 pthread_mutex_lock /home/kito/llvm/src/projects/compiler- rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:2956 (race- 2+0x00000007f260) #1 Thread1 /home/kito/coscup2014/race-2.c:8 (race- 2+0x0000000c1a37) Thread T2 (tid=21768, running) created by main thread at: #0 pthread_create /home/kito/llvm/src/projects/compiler- rt/lib/tsan/rtl/tsan_interceptors.cc:842 (race-2+0x00000005e231) #1 main /home/kito/coscup2014/race-2.c:22 (race-2+0x0000000c1b63) Thread T1 (tid=21767, finished) created by main thread at: #0 pthread_create /home/kito/llvm/src/projects/compiler- rt/lib/tsan/rtl/tsan_interceptors.cc:842 (race-2+0x00000005e231) 55 GCC ! 56 class T { public: int a; } #include 57 class T { public: int a; } #include In file included from /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64- unknown-linux-gnu/4.4.7/../../../../include/c++/4.4.7/cstddef:44, from /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64- unknown-linux-gnu/4.4.7/../../../../include/c++/4.4.7/bits/stl_algobase.h:61, from /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64- unknown-linux-gnu/4.4.7/../../../../include/c++/4.4.7/vector:61, from test.cpp:5: /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64-unknown-linux- gnu/4.4.7/include/stddef.h:149: error: two or more data types in declaration of ptrdiff_t ... /home/kito/gcc-workspace/native-4.4/lib/gcc/x86_64-unknown-linux- gnu/4.4.7/../../../../include/c++/4.4.7/bits/vector.tcc:629: error: there are no arguments to difference_type that depend on a template parameter, so a declaration of difference_type must be available gcc 4.4 132 XD 58 class T { public: int a; } #include gcc 4.9 test.cpp:4:1: error: expected ; after class definition } ^ ( clang ? 59 : http://web.archive.org/web/20120622065456/http://people .redhat.com/bkoz/diagnostics/diagnostics.html https://gcc.gnu.org/wiki/ClangDiagnosticsComparison http://tinyurl.com/cxxdiagcmp http://tinyurl.com/clangcmp 60 * Sanitizer Clang/LLVM GCC 4.8 ! 61 LTO ! Firefox Linux Kernel 62 libgccjit.so gcc JIT engine !! 63 libgccjit.so gcc JIT engine !! : 64 libgccjit.so gcc JIT engine !! : gcc + dlopen 65 C++ 66 LLVM ! 67 GCC 68 Named register variables register int *foo asm ("a5"); 69 Named register variables , Inline asm register int *foo asm ("a5"); 70 Named register variables , Inline asm Linux Kernel register int *foo asm ("a5"); 71 Named register variables , Inline asm Linux Kernel Trunk, 3.5 register int *foo asm ("a5"); 72 OpenMP Support int main(int argc, char *argv[]) { const int N = 100000; int i, a[N]; #pragma omp parallel for for (i = 0; i < N; i++) a[i] = 2 * i; return 0; } 73 1.x 2.x 3.x 74 1.x 2.x 3.x bug fix back port 75 1.x 2.x 3.x bug fix back port bug? Fix in ToT! 76 1.x 2.x 3.x bug fix back port bug? Fix in ToT! 3.4 3.4.1, 3.4.2! 77 gcc ! http://vmakarov.fedorapeople.org/spec/2014/llvmgcc64.html 78 79 -O -I -l -D -u -fPIC -Wl, -Wa, ... Open Source Project gcc 80 Clang (~3.4.x) Warning, return code 81 Clang (~3.4.x) Warning, return code Clang (trunk) Error abort! 82 Warning -Wall -Werror -Wextra clang gcc ... 83 Code !? Code gcc ! 84 Code !? Code gcc ! gcc/g++ gnu89/ gnu++98 c89/c++98 GNU extension 85 Code !? Code gcc ! gcc/g++ gnu89/ gnu++98 c89/c++98 GNU extension clang c99! GNU extension 86 GCC and LLVM collaboration 87 GCC and LLVM collaboration GNU Tools Cauldron 2014 July 18-20, 2014 at Cambridge 88 GCC and LLVM collaboration With LLVM mature enough to feature as the default toolchain in some Unix distributions, and with the inherent (and profitable) share of solutions, ideas and code between the two, we need to start talking at a more profound level. There will always be problems that can't be included in any standard (language, extension, or machine-specific) and are intrinsic to the compilation infrastructure. For those, and other common problems, we need common solutions to at least both LLVM and GCC, but ideally any open source (and even closed source) toolchain. In this BoF session, we shall discuss to what extent this collaboration can take us, how we should start and what are the next steps to make this happen. 89 90 91 Open Source++ 92 Toolchain ~ Open Source++ 93