Distro Recipes 2013 : Make Debian and compiler agnostic

Download Distro Recipes 2013 : Make Debian and  compiler agnostic

Post on 11-May-2015

2.806 views

Category:

Technology

2 download

Embed Size (px)

DESCRIPTION

https://distro-recipes.org

TRANSCRIPT

<ul><li>1.Make Debian compiler agnostic :A rebuild with ClangSylvestre Ledru sylvestre@debian.org</li></ul> <p>2. Disclamer !April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 3. Current status : All C, C++, Objective-C sources are being built with gcc for all supported Debian arches and Kernel.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 4. gcc is THE FLOSS compiler for the last 25 years Used for (pretty much) everywhere or anythingStill growing and maintained (cf gcc 4.8)April, 04 2013Make Debian compiler agnosticSylvestre Ledru 5. Why a new compiler in Debian ?April, 04 2013Make Debian compiler agnosticSylvestre Ledru 6. Other compilers can find programming errors thatgcc could not findApril, 04 2013Make Debian compiler agnosticSylvestre Ledru 7. Code built by many compilers is more likely tobe more strictly correct and more portable thencode only built with gccApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 8. Some compilers can have advantages on some archs (ex : clang on ARM)April, 04 2013Make Debian compiler agnosticSylvestre Ledru 9. Provide some interesting side projects (lldb, scan- build, ASAN, libc++, etc)April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 10. As we were able to do with decoupling Linuxfrom Debian with kFreeBSD and the HURD, were aiming to decouple gcc in Debian. Open the door to other compilers or analyzersApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 11. LLVM/ClangApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 12. Started as an academic project Versatile platform for compilation and virtual machine Designed originally for the investigation of dynamic compilation techniques for static and dynamic languagesApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 13. Developed initially as a base for a next majorGCC release Source: http://gcc.gnu.org/ml/gcc/2005-11/msg00888.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 14. Sponsored by Apple since 2005 to replace gcc(GPL vs BSD)Has now a strong and diverse community(academics, individuals and corporates) Many universities/research centers are basing their research on LLVMApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 15. April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 16. Clang C, C++ &amp; Objective-C compiler.(no Fortran)Based on LLVMDefault compiler for Mac OS X (Xcode)/iOS [1]and FreeBSD [2] Sources: [1] https://developer.apple.com/technologies/tools/ [2] http://lists.freebsd.org/pipermail/freebsd-stable/2012-May/067486.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 17. Some advantages : More recent base code (ie less legacy code) Strong interest of hardware manufactors (ARM, MIPS, Intel, Nvidia, etc)Supposed to be faster to build code than gcc Accept the same arguments as gccApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 18. ExampleFull build of Scilab (doc, essential tests) ~24 minutes gcc~22 minutes clangApril, 04 2013Make Debian compiler agnosticSylvestre Ledru 19. Some advantages (bis)More intelligent detections foo.c $ gcc -Wall -Werror foo.c ; echo $?int main() {0unsigned int i = 0; $ clang -Werror foo.cfoo.c:3:17: error: comparison ofreturn i &lt; 0;unsigned expression &lt; 0 is always} false [-Werror,-Wtautological-compare]return i &lt; 0;~^~1 error generated.April, 04 2013Make Debian compiler agnosticSylvestre Ledru 20. Side effect=&gt; Brings (friendly) competition in the freecompiler world.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 21. http://gcc.gnu.org/wiki/ClangDiagnosticsComparisonApril, 04 2013Make Debian compiler agnosticSylvestre Ledru 22. http://clang.llvm.org/comparison.html#gccApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 23. Rebuild of Debian using ClangApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 24. Crappy method :VERSION=4.7cd /usr/binrm g++-$VERSION gcc-$VERSION cpp-$VERSIONln -s clang++ g++-$VERSIONln -s clang gcc-$VERSIONln -s clang cpp-$VERSIONcd -April, 04 2013Make Debian compiler agnosticSylvestre Ledru 25. Testing the rebuild of the package under amd64. NOT the performances (build time or execution) nor the execution of the binariesApril, 04 2013Make Debian compiler agnosticSylvestre Ledru 26. Full results published: http://clang.debian.net/Done on the cloud-qa - EC2 (Amazon cloud) Thanks to Lucas NussbaumApril, 04 2013Make Debian compiler agnosticSylvestre Ledru 27. Rebuild with clang 3.0February 28, 2012 15658 packages built : 1381 (8.8 %) failed.April, 04 2013Make Debian compiler agnosticSylvestre Ledru 28. Rebuild with clang 3.1June 23, 2012 17710 packages built : 2137 (12.1 %) failed.April, 04 2013Make Debian compiler agnosticSylvestre Ledru 29. Rebuild with clang 3.2 January 28, 201318264 packages built : 2204 (12.1 %) failed.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 30. Why these differences between 3.0 vs 3.1/3.2?April, 04 2013Make Debian compiler agnosticSylvestre Ledru 31. Some information about -Wall &amp; -Werror : -Wall enables many warnings -Werror transforms Warning to Errorint main() {$ gcc -Wall -Werror foo.c &amp;&amp; echo $? unsigned int i = 0;0 return i &lt; 0;} $ clang -Wall -Werror foo.c &amp;&amp; echo $?foo.c:3:14: error: comparison of unsigned expression &lt; 0 is always false[-Werror,-Wtautological-compare] return i &lt; 0;~^~1 error generated.April, 04 2013Make Debian compiler agnosticSylvestre Ledru 32. -Werror &amp; unused args 40 occurrencesClang detects unused argument.clang --param ssp-buffer-size=4 clang: error: argument unused-Werror foo.c during compilation: --param ssp-buffer-size=4And generates a normal warning 96 occurrencesWhich becomes an error with -Werror April, 04 2013 Make Debian compiler agnosticSylvestre Ledru 33. Security check introduced in clang 3.1 9 occurences $ gcc -Werror -c foo.c &amp;&amp; echo $?#include 0void foo(void) { $ clang -Werror -c foo.c &amp;&amp; echo $?char buffer[1024]; foo.c:5:23: error: use of %n insprintf(buffer, "%n", 2);format string discouraged(potentially insecure) [-Werror,-} Wformat-security] sprintf(buffer, "%n", 2); ~^ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 34. Some of the most common errorsApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 35. Unsupported options50 occurrences$ gcc -O9 foo.c &amp;&amp; echo $?0$ clang -O9 foo.cerror: invalid value 9 in -O9Record by libdbi-drivers with -O20 o/April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 36. Different default behavior 133 occurrences noreturn.c $ gcc -c noreturn.c; echo $?int foo(void) {0 # -Wall shows it as warningreturn; $ clang -c noreturn.c} noreturn.c:2:2: error: non-void function foo should return a value[-Wreturn-type]return;^ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 37. Different default behavior (bis)15 occurrences$ gcc -c returninvoid.c; echo $? returninvoid.c returninvoid.c: In function foo:void foo(void) {returninvoid.c:2:2: warning: return with areturn 42;value, in function returning void [enabledby default]}0$ clang -c returninvoid.creturninvoid.c:2:2: error: void functionfoo should not return a value [-Wreturn-type] return 42; ^~~1 error generated.April, 04 2013Make Debian compiler agnosticSylvestre Ledru 38. Different understanding of the C++ standard mailboxField.cpp $ g++ -c mailboxField.cpp &amp;&amp; echo $?class address {0protected:static int parseNext(int a);$ clang++ -c mailboxField.cpp};mailboxField.cpp:17:22: error: parseNext is aclass mailbox : public address {protected member of addressfriend class mailboxField; address::parseNext(a);};^class mailboxField {mailboxField.cpp:4:16: note: declared protectedvoid parse(int a){here address::parseNext(a); static int parseNext(int a); // will work with:^ // mailbox::parseNext(a);}References:};http://llvm.org/bugs/show_bug.cgi?id=6840http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52136April, 04 2013Make Debian compiler agnosticSylvestre Ledru 39. Different set of warnings with -WallPlenty of occurences echo $?$ gcc -Wall -Werror -c foo.cpp &amp;&amp; plop.c 0void foo() { $ clang -Wall -Werror -c foo.cpp foo.cpp:3:13: error: equality comparison with extraneousint a=1; parenthesesif ((a == 1)) {[-Werror,-Wparentheses-equality] if ((a == 1)) {return; ~~^~~~ foo.cpp:3:13: note: remove extraneous parentheses around the}comparison to}silence this warning if ((a == 1)) { ~ ^ ~ foo.cpp:3:13: note: use = to turn this equality comparison into an assignment if ((a == 1)) {^~=1 error generated.April, 04 2013Make Debian compiler agnosticSylvestre Ledru 40. gcc extensions which wont be supported 25 occurences foo.cpp $ g++ -c foo.cpp; echo $?#include 0 $ clang++ -c foo.cppvoid foo() { foo.cpp:4:29: error: variable lengthint N=2; array of non-POD element typestd::vector best[2][N];std::vector}std::vector best[2][N]; ^ 1 error generated.April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 41. gcc accepts stuff which should not34 occurences$ g++ -c foo.cpp; echo $? foo.cpp 0// Uncomment this line will fix the issue.$ clang++ -c foo.cpp// templatefoo.cpp:6:5: error: call to function b that is neither// void b(Value_t value)visible in the templatetemplatedefinition nor found by argument-dependent lookupvoid a(Value_t value) {b(value);b(value);^}foo.cpp:15:5: note: in instantiation of function templatespecializationa requested heretemplatevoid b(Value_t value) { a(y);} ^foo.cpp:9:33: note: b should be declared prior to the call sitetemplate void b(Value_t value)void foo(int y) {^a(y);1 error generated.}April, 04 2013Make Debian compiler agnosticSylvestre Ledru 42. Last rebuild proved that clang is now ready Remaining problems are upstreamSome work started to report bugs upstreamApril, 04 2013Make Debian compiler agnosticSylvestre Ledru 43. StatusApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 44. Packages LLVM-toolchain containing LLVM + Clang + compiler-rt + polly + lldb version 3.2 &amp; 3.3 waiting in Debian NEWApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 45. Build results integrated on the Debian Package Tracking System By Paul WiseApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 46. Version of pbuilder/cowbuilder with a hook tochange the compiler By Junich UekawaExemple : sudo CHOOSE_COMPILER=clang cowbuilder --build yourpackage_1.00-1.dscApril, 04 2013Make Debian compiler agnosticSylvestre Ledru 47. Interest by upstreams (Google, Intel &amp; Apple)Publication of Debian (wheezy, unstable) and Ubuntu (Quantal, Precise, Raring) of Nightlypackages of LLVM, Clang, Compiler-rt, Polly &amp;LLDBhttp://llvm.org/apt http://blog.llvm.org/2013/04/llvm-debianubuntu-nightly-packages.htmlApril, 04 2013Make Debian compiler agnosticSylvestre Ledru 48. FutureApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 49. Discussions to move the build as a new port on http://www.debian-ports.org/ Various potential issues (not really new arch, mix of gcc/clang-built packages, etc)April, 04 2013Make Debian compiler agnosticSylvestre Ledru 50. Hack the Debian tools to :Force dpkg to export CC=/usr/bin/cc, CXX=/usr/bin/c++ and OBJC=/usr/bin/objc Check for hardcoded CC=gcc in debian/rules Set the /usr/bin/{cc,c++,objc} alternatives to{clang, clang++}April, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 51. Update the debian policy to include something like : Hardcoded usage of CC or CXX (for example, CC=gcc) should be avoid and documented if necessary. Debian build tools must respect the CC and CXX variables ifprovided. If not, they shall default to /usr/bin/cc and /usr/bin/c++See : http://lists.debian.org/debian-devel/2012/08/msg00783.htmlApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 52. Add a lintian warning like W: yourpackage: Hardcoded call to gcc/g++.Use /usr/bin/cc or /usr/bin/c++ insteadApril, 04 2013 Make Debian compiler agnostic Sylvestre Ledru 53. Potential the rebuild of Debian with :clang+plugin. Ex : polly : cache-localityoptimisation auto-parallelism and vectorization, etc address sanitizer (ASAN) scan-build : static C/C++ analyzer Intel compilersApril, 04 2013Make Debian compiler agnosticSylvestre Ledru 54. Any questions ? Remarks ?Troll ? (+1)April, 04 2013Make Debian compiler agnosticSylvestre Ledru </p>