inlining java native calls at runtime - university of albertaamaral/cascon/cdp... · inlining java...
TRANSCRIPT
![Page 1: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/1.jpg)
Inlining Java Native Calls at Runtime(CASCON 2005 – 4th Workshop on Compiler Driven Performance)
Levon Stepanian, Angela Demke BrownComputer Systems Group
Department of Computer Science, University of Toronto
Allan Kielstra, Gita Koblents, Kevin StoodleyIBM Toronto Software Lab
![Page 2: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/2.jpg)
In a nutshell• Runtime native function inlining into Java
• Optimizing transformations on inlined JNI calls• Opaque and binary-compatible while boosting
performance
Java Code
Native CodeNative Function Call
![Page 3: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/3.jpg)
In a nutshell• Runtime native function inlining into Java
• Optimizing transformations on inlined JNI calls• Opaque and binary-compatible while boosting
performance
Java Code
Native Codeinlined
![Page 4: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/4.jpg)
In a nutshell• Runtime native function inlining into Java
• Optimizing transformations on inlined JNI calls• Opaque and binary-compatible while boosting
performance
Java Code
Native Codeinlined
+optimized
![Page 5: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/5.jpg)
Motivation• The JNI
• Java’s interoperability API• Callouts and callbacks• Opaque• Binary-compatible
Java App
Native (Host) Environment
Native App/Lib
JVM + JIT
CalloutCallback
JNI
![Page 6: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/6.jpg)
Motivation• The JNI
• Pervasive• Legacy codes• Performance-critical, architecture-dependent• Features unavailable in Java (files, sockets
etc.)
![Page 7: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/7.jpg)
Motivation
• Callouts run to 2 to 3x slower than Java calls• Callback overheads are an order of magnitude larger
• JVM handshaking requirements for threads leaving and re-entering JVM context
• i.e. stack switching, reference collection, exception handling
• JIT compiler can’t predict side-effects of native function call
![Page 8: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/8.jpg)
Our Solution• JIT compiler based optimization that inlines
native code into Java• JIT compiler transforms inlined JNI
function calls to constants, cheaper operations
• Inlined code exposed to JIT compiler optimizations
![Page 9: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/9.jpg)
Infrastructure• IBM TR JIT Compiler + IBM J9 VM• Native IL to JIT IL conversion mechanism
• Exploit Native IL stored in native libraries • W-Code to TR-IL at runtime
TR JITMachine
codeStatic
compiler IL
+
![Page 10: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/10.jpg)
Outline• Background Information ➼• Method• Results• Future Work
![Page 11: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/11.jpg)
Sample Java Classclass SetFieldXToFive{
public int x; public native foo();
static{ System.loadLibrary(…); }}
![Page 12: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/12.jpg)
Sample Java Classclass SetFieldXToFive{
public int x; public native foo();
static{ System.loadLibrary(…); }}
![Page 13: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/13.jpg)
Sample Native Code
JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){
jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;
(*env)->SetIntField(env,obj,fid,5);}
GOAL: obj.x = 5
![Page 14: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/14.jpg)
Sample Native Code
JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){
jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;
(*env)->SetIntField(env,obj,fid,5);}
GOAL: obj.x = 5
SetFieldXToFive
![Page 15: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/15.jpg)
Sample Native Code
JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){
jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;
(*env)->SetIntField(env,obj,fid,5);}
GOAL: obj.x = 5
![Page 16: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/16.jpg)
Sample Native Code
JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){
jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;
(*env)->SetIntField(env,obj,fid,5);}
GOAL: obj.x = 5
![Page 17: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/17.jpg)
Sample Native Code
JNIEXPORT void JNICALL Java_SetFieldXToFive_foo (JNIEnv * env, jobject obj){
jclass cls = (*env)->GetObjectClass(env,obj); jfieldID fid = (*env)->GetFieldID(env,cls,“x","I"); if (fid == NULL) return;
(*env)->SetIntField(env,obj,fid,5);}
GOAL: obj.x = 5
![Page 18: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/18.jpg)
Native Inlining Overview
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
3. Identifies inlined JNI calls
4. Transforms inlined JNI calls
5. Finishes inlining
![Page 19: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/19.jpg)
Method – Step 1
1. Inliner detects a native callsiteInliner
Java Code
Call to obj.foo()
foo(){…}(Native code)
TR JIT
![Page 20: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/20.jpg)
Method – Step 2
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
Native IL
JIT IL
![Page 21: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/21.jpg)
Method – Step 3
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
3. Identifies inlined JNI calls
JIT IL
/* call to GetObjectClass */
…
/* call to GetFieldID */
…
/* call to SetFieldID */
…
Pre-constructed IL shapes
![Page 22: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/22.jpg)
Method – Step 4
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
3. Identifies inlined JNI calls
4. Transforms inlined JNI calls
jclass cls = (*env)->GetObjectClass(env,obj);
jfieldID fid = (*env)->GetFieldID(env,cls,“x","I");if (fid == NULL) return;
(*env)->SetIntField(env,obj,fid,5);
![Page 23: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/23.jpg)
Method – Step 4
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
3. Identifies inlined JNI calls
4. Transforms inlined JNI calls
(*env)->SetIntField(env,obj,fid,5);
Constant: SetFieldXToFive class data structure
jfieldID fid = (*env)->GetFieldID(env,cls,“x","I");if (fid == NULL) return;
![Page 24: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/24.jpg)
Method – Step 4
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
3. Identifies inlined JNI calls
4. Transforms inlined JNI calls
(*env)->SetIntField(env,obj,fid,5);
Constant: SetFieldXToFive class data structure
Constant: Offset of field “x”
![Page 25: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/25.jpg)
Method – Step 4
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
3. Identifies inlined JNI calls
4. Transforms inlined JNI calls
Constant: SetFieldXToFive class data structure
Constant: Offset of field “x”
JIT IL: obj.x = 5
![Page 26: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/26.jpg)
The Big Picture
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
3. Identifies inlined JNI calls
4. Transforms inlined JNI calls
5. Finishes inlining
Before Native Inlining & Callback
Transformations
Inliner
Java Code
Call to obj.foo()
TR JIT
foo(){…}(Native code)
![Page 27: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/27.jpg)
The Big PictureAfter Native Inlining & Callback
Transformations
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
3. Identifies inlined JNI calls
4. Transforms inlined JNI calls
5. Finishes inlining
Inliner
Java Code
obj.x = 5
TR JIT
foo(){…}(Native code)
![Page 28: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/28.jpg)
The Big PictureAfter Native Inlining & Callback
Transformations
1. Inliner detects a native callsite
2. Extracts and converts Native IL to JIT IL
3. Identifies inlined JNI calls
4. Transforms inlined JNI calls
5. Finishes inlining
Inliner
Java Code
obj.x = 5
TR JIT
![Page 29: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/29.jpg)
Outline• Background Information ➼• Method ➼• Results• Future Work
![Page 30: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/30.jpg)
Experimental Setup• Native function microbenchmarks
• Average of 300 million runs• 1.4 GHz Power4 setup• Prototype implementation
![Page 31: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/31.jpg)
Cost of IL Conversion• 5.3 microseconds per W-Code
0
1
2
3
4
5
6
7
bzip2
crafty ga
pgcc
gzip mc
f
parser
perlbmk tw
olf
vortex
vpr
SPEC CINT2000 Benchmarks
Tim
e pe
r Opc
ode
(mic
rose
cs.)
![Page 32: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/32.jpg)
Inlining Null Callouts• Null native method microbenchmarks• Varying numbers of args (0, 1, 3, 5)
• Complete removal of call/return overhead• Gain back 2 to 3x slowdown
• confirmed our expectations
![Page 33: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/33.jpg)
Inlining Non-Null Callouts
1.85.5hashStaticInstanceMicrobenchmark Test
Speedup (X)
•smaller speedups for natives performing work
•instance vs. static speedup
![Page 34: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/34.jpg)
Inlining & Transforming Callbacks
•Reclaim order of magnitude overhead
11.812.9CallVoidMethodStaticInstanceMicrobenchmark Test
Speedup (X)
![Page 35: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/35.jpg)
Data-Copy Speedups
• Transformed GetIntArrayRegionArray Length
Spee
dup
(X)
![Page 36: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/36.jpg)
Exposing Inlined Code To JIT Optimizations
93.4GetArrayLengthSpeedup (X)Microbenchmark Test
FindClassGetMethodIDNewCharArrayGetArrayLength
![Page 37: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/37.jpg)
Conclusion• Runtime native function inlining into Java code• Optimizing transformations on inlined Java Native
Interface (JNI) calls• JIT optimize inlined native code• Opaque and binary-compatible while boosting
performance
• Future Work• Engineering issues• Heuristics• Larger interoperability framework
![Page 38: Inlining Java Native Calls at Runtime - University of Albertaamaral/cascon/CDP... · Inlining Java Native Calls at Runtime (CASCON 2005 – 4th Workshop on Compiler Driven Performance)](https://reader034.vdocuments.mx/reader034/viewer/2022042307/5ed2de36a079355bb26da2ab/html5/thumbnails/38.jpg)
Fin