Predicting Good Compiler ?· Predicting Good Compiler Transformations Using Machine Learning Edwin V.…

Download Predicting Good Compiler ?· Predicting Good Compiler Transformations Using Machine Learning Edwin V.…

Post on 30-Jul-2018

215 views

Category:

Documents

2 download

Embed Size (px)

TRANSCRIPT

<ul><li><p>Predicting Good Compiler Transformations </p><p>Using Machine Learning </p><p>Edwin V. Bonilla </p><p>Master of Science </p><p>Artificial Intelligence </p><p>School of Informatics </p><p>University of Edinburgh </p><p>2004</p></li><li><p> I</p><p>Abstract </p><p>This dissertation presents a machine learning solution to the compiler optimisation problem </p><p>focused on a particular program transformation: loop unrolling. Loop unrolling is a very </p><p>straightforward but powerful code transformation mainly used to improve Instruction Level </p><p>Parallelism and to reduce the overhead due to loop control. However, loop unrolling can also </p><p>be detrimental, for example, when the instruction cache is degraded due to the size of the </p><p>loop body. Additionally, the effect of the interactions between loop unrolling and other </p><p>program transformations is unknown. Consequently, determining when and how unrolling </p><p>should be applied remains a challenge for compiler writers and researchers. This project </p><p>works under the assumption that the effect of loop unrolling on the execution times of </p><p>programs can be learnt based on past examples. Therefore, a regression approach able to </p><p>learn the improvement in performance of loops under unrolling is presented. This novel </p><p>approach differs from previous work ([Monsifrot et al., 2002] and [Stephenson and </p><p>Amarasinghe, 2004]) because it does not formulate the problem as a classification task but as </p><p>a regression solution. Great effort has been invested in the generation of clean and reliable </p><p>data in order to make it suitable for learning. Two different regression algorithms have been </p><p>used: Multiple Linear Regression and Classification and Regression Trees (CART). </p><p>Although the accuracy of the methods is questionable, the realisation of final speed-ups on </p><p>seven out of twelve benchmarks indicates that something has been gained with this learning </p><p>process. A maximum 18% of re-substitution improvement has been achieved. 2.5% of </p><p>overall improvement in performance for Linear Regression and 2.3% for CART algorithm </p><p>have been obtained. The present work is the beginning of an ambitious project that attempts </p><p>to build a compiler that can learn to optimise programs and can undoubtedly be improved in </p><p>the near future. </p></li><li><p> II</p><p>Acknowledgements Special thanks to my supervisor Dr. Chris Williams for his invaluable advice and </p><p>comprehensive revision of my progress throughout this project. </p><p>Thanks to Dr. Michael O'Boyle and Dr. Grigori Fursin for the discussions held during our </p><p>meetings that made possible the creation of the dataset that has been used in this project. </p><p>Thanks to Catalina Voroneanu and Bonny Quick for their patience when revising some </p><p>drafts of this dissertation. </p><p>Supported by the Programme Alan, European Union Programme of High Level </p><p>Scholarships for Latin America, identification number (E03M14650CO). </p></li><li><p> III</p><p>Declaration I declare that this thesis was composed by myself, that the work contained herein is my own </p><p>except where explicitly stated otherwise in the text, and that this work has not been </p><p>submitted for any other degree or professional qualification except as specified. </p><p>(Edwin V. Bonilla) </p></li><li><p> IV</p><p>Table of Contents </p><p>Introduction.............................................................................................................................. 1 Overview and Motivation .................................................................................................... 1 Project Objectives ................................................................................................................ 2 Organisation......................................................................................................................... 3 </p><p>Chapter One: Literature Review .............................................................................................. 6 1.1 Introduction ............................................................................................................ 6 1.2 Tuning heuristics and recommending program transformations ............................ 6 1.3 Learning in a particular program transformation: loop unrolling........................... 8 1.4 Summary............................................................................................................... 10 </p><p>Chapter 2: Background on Compiler Optimisation ............................................................... 11 2.1 Introduction .......................................................................................................... 11 2.2 Definition of compilation ..................................................................................... 11 2.3 Compiler organisation .......................................................................................... 12 2.4 The purpose of a compiler .................................................................................... 13 2.5 An Optimising Compiler ...................................................................................... 14 </p><p>2.5.1 Goals of Compiler Optimisation ...................................................................... 14 2.5.2 Considerations for program transformations.................................................... 15 2.5.3 The process of transforming a program for optimisation................................. 16 2.5.4 The problem of interaction............................................................................... 17 2.5.5 Types of program transformations ................................................................... 18 2.5.6 The scope of optimisation ................................................................................ 18 2.5.7 Some common transformations........................................................................ 20 </p><p>2.6 Loop Unrolling ..................................................................................................... 20 2.6.1 Definition ......................................................................................................... 20 2.6.2 Implementation ................................................................................................ 21 2.6.3 Advantages of loop unrolling........................................................................... 22 2.6.4 Disadvantages of loop unrolling ...................................................................... 23 2.6.5 Interactions, again ............................................................................................ 24 2.6.6 Candidates for unrolling................................................................................... 24 </p><p>2.7 Summary............................................................................................................... 24 Chapter 3: Data Collection..................................................................................................... 26 </p><p>3.1 Introduction .......................................................................................................... 26 </p></li><li><p> V</p><p>3.2 The Benchmarks................................................................................................... 28 3.3 Implementation of loop unrolling......................................................................... 31 </p><p>3.3.1 Which loops should be unrolled? ..................................................................... 31 3.3.2 Initial experiments............................................................................................ 31 3.3.3 Loop level profiling.......................................................................................... 32 </p><p>3.4 Generating the targets........................................................................................... 34 3.4.1 Preparing the benchmarks ................................................................................ 34 3.4.2 Selecting loops ................................................................................................. 34 3.4.3 Profiling............................................................................................................ 34 3.4.4 Filtering............................................................................................................ 34 3.4.5 Running the search strategy ............................................................................. 35 </p><p>3.5 Technical Details .................................................................................................. 35 3.5.1 The platform..................................................................................................... 35 3.5.2 The compiler .................................................................................................... 35 3.5.3 The timers precision........................................................................................ 35 </p><p>3.6 The results in summary......................................................................................... 36 3.7 Feature extraction ................................................................................................. 36 3.8 The representation of a loop ................................................................................. 39 3.9 Summary............................................................................................................... 40 </p><p>Chapter 4: Data Preparation and Exploratory Data Analysis................................................. 41 4.1 Introduction .......................................................................................................... 41 4.2 The general framework for data integration ......................................................... 42 4.3 Formal representation of the data ......................................................................... 43 4.4 Is this data valid? .................................................................................................. 44 </p><p>4.4.1 Statistical analysis ............................................................................................ 44 4.5 Pre-processing the targets..................................................................................... 46 </p><p>4.5.1 Filtering............................................................................................................ 48 4.5.2 Dealing with outliers ........................................................................................ 49 4.5.3 Target transformation....................................................................................... 50 </p><p>4.6 Pre-processing the features................................................................................... 52 4.6.1 Rescaling.......................................................................................................... 52 4.6.2 Feature selection and feature transformation ................................................... 53 </p><p>4.7 Summary............................................................................................................... 54 Chapter 5: Modelling and Results.......................................................................................... 57 </p><p>5.1 Introduction .......................................................................................................... 57 </p></li><li><p> VI</p><p>5.2 The regression approach....................................................................................... 58 5.3 Learning methods used......................................................................................... 60 </p><p>5.3.1 Multiple Linear Regression.............................................................................. 62 5.3.2 Classification and Regression Trees ................................................................ 64 </p><p>5.4 Parameters setting................................................................................................. 64 5.5 Measure of performance used............................................................................... 65 5.6 Experimental Design ............................................................................................ 65 </p><p>5.6.1 Complete dataset .............................................................................................. 65 5.6.2 K-fold cross-validation..................................................................................... 66 5.6.3 Leave One Benchmark Out cross-validation.................................................... 66 5.6.4 Realising speed-ups.......................................................................................... 67 </p><p>5.7 Results and Evaluation ......................................................................................... 67 5.7.1 Complete dataset .............................................................................................. 67 5.7.2 K-fold cross-validation..................................................................................... 70 5.7.3 Leave One Benchmark Out Cross-validation................................................... 71 5.7.4 Realising speed-ups.......................................................................................... 75 5.7.5 Feature construction ......................................................................................... 78 5.7.6 Comparison to related work ............................................................................. 78 </p><p>5.8 Summary and Discussion ..................................................................................... 79 Conclusions............................................................................................................................ 87 Bibliography .......................................................................................................................... 91 </p></li><li><p> 1</p><p>Introduction </p><p>Overview and Motivation The continuously increasing demand of high performance computational resources has </p><p>required great effort on behalf of hardware designers in order to develop advanced </p><p>components making possible the construction of current applications. Thus, </p><p>microprocessors architectures have become more complex and have come to provide </p><p>processing rates that until a few years ago were affordable only by a selected group of users. </p><p>However, the resources offered by current processors are significantly underexploited, which </p><p>limits the possibility of running applications at maximum speed. </p><p>This bottleneck is mainly a consequence of the limitations of a very complex application </p><p>being used to transform the source code of programs written in a high-level programming </p><p>language into machine dependent code, the compiler. Indeed, the main concern of compiler </p><p>writers centres not upon very well studied tasks at the front of the compilation process (such </p><p>as parsing and lexical analysis) but upon the discovery of ways in which to optimise the </p><p>execution time of programs. Hence, optimisation is not considered an additional feature of </p><p>the compilation process but has become a crucial component of existing compilers. </p><p>However, achieving high performance on modern processors is an extremely difficult ta...</p></li></ul>