Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
02/22/2013
1
Welcome to the session...
Monday, September 30, 13
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
02/22/2013
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any
material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any
features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
© 2013 Oracle Corporation2
Basically, don’t trust anything I say...
Monday, September 30, 13
Rickard BäckmanHotSpot JVM Compiler Team, Java Platform
Java Flight Recorder - Next generation diagnostics and profiling
Monday, September 30, 13
© 2013 Oracle Corporation4
public class Example { public void saveTransaction(Transaction tn) { synchronized (someLock) { // save to some shared data structure } }}
Monday, September 30, 13
© 2013 Oracle Corporation5
public class Example { public void saveTransaction(Transaction tn) { System.err.println(threadId + “Waiting for lock”); synchronized (someLock) { System.err.println(threadId + “Got lock”); // save to some shared data structure } System.err.println(threadId + “Lock released”); }}
Monday, September 30, 13
© 2013 Oracle Corporation6
... few million lines of output[T1] Waiting for lock[T3] Waiting for lock[T2] Waiting for lock[T2] Got lock[T1] Got lock[T1] Lock released[T3] Got lock[T3] Lock released[T2] Lock released... another million lines of output
Monday, September 30, 13
© 2013 Oracle Corporation
Java Flight Recorder - Overview
7
Released in 7u40
Monday, September 30, 13
© 2013 Oracle Corporation
Java Flight Recorder - Overview
Event Recorder & Profiler
9
Monday, September 30, 13
© 2013 Oracle Corporation
Java Flight Recorder - Overview
Implemented in the JVM
10
Monday, September 30, 13
© 2013 Oracle Corporation
Java Flight Recorder - Overview
No restart required
11
Monday, September 30, 13
© 2013 Oracle Corporation
Java Flight Recorder - Overview
12
After-the-facts analysis-
What went wrong?
Monday, September 30, 13
Instrumented JVM & JDK-
Make that information visible
© 2013 Oracle Corporation
Event Recorder
15
Monday, September 30, 13
Core in the JVMHigh-level things in Java
© 2013 Oracle Corporation
Inside the JVM
17
Monday, September 30, 13
Access toJVM internals & subsystems
© 2013 Oracle Corporation
Inside the JVM
18
Monday, September 30, 13
Designed to run all the time.In your production environment
© 2013 Oracle Corporation
Non-intrusive
20
Monday, September 30, 13
Low overheadCaptures information that is already there
© 2013 Oracle Corporation
Non-intrusive
21
Monday, September 30, 13
Java Mission Control-
Dump the data on an SLA breach
© 2013 Oracle Corporation
After-the-facts
26
Monday, September 30, 13
© 2013 Oracle Corporation
Things JFR captures
27
Garbage CollectionMethod Compilation
Threads
LocksProfilingCodeCache
Class Load
Class Unload
Object allocation
Exceptions
Metaspace Statistics
Context switches
CPU Load
Object CountVM OperationsSocket read/write
Thread.sleepFile read/writeGarbage Collection statistics
Compilation statistics
Monday, September 30, 13
How to implement a profiler?
© 2013 Oracle Corporation
Sampling Profiler
28
Monday, September 30, 13
Stop the thread,walk the stack,
collect the methods.
© 2013 Oracle Corporation
Sampling Profiler
31
Monday, September 30, 13
-XX:+UnlockCommercialFeatures-XX:+FlightRecorder
© 2013 Oracle Corporation
Enable Java Flight Recorder
33
Monday, September 30, 13
-XX:StartFlightRecording=filename=<...>,duration=5m
© 2013 Oracle Corporation
Start recording
34
Monday, September 30, 13
Java Mission Controlor
jcmd <pid> JFR.start <options>
© 2013 Oracle Corporation
Start on demand
35
Monday, September 30, 13
stackdepth=<nr>
dumponexit=<true/false>
© 2013 Oracle Corporation
Some interesting options
36
Monday, September 30, 13
Captures something interestingLatency? Performance?
© 2013 Oracle Corporation
Events
38
Monday, September 30, 13
Happens “instantly”-
Thread start, Thread stop, Class unload
© 2013 Oracle Corporation
Instant Events
39
Monday, September 30, 13
Happens over a period of time-
Compilation, Garbage Collection,Wait for a lock
© 2013 Oracle Corporation
Duration Events
40
Monday, September 30, 13
Something that should be done repeatedly-
CPU Load, Profiling EventsContext switches
© 2013 Oracle Corporation
Requestable Events
41
Monday, September 30, 13
Applies a filter and captures the remaining
© 2013 Oracle Corporation
Recording
50
Monday, September 30, 13
A preconfigured settings for a recording
© 2013 Oracle Corporation
Profiles
54
Monday, September 30, 13
Implementation
© 2013 Oracle Corporation
Java Flight Recorder
58
Challenges
Events
Memory buffers
File format
Assembly!
C++
Low level
Monday, September 30, 13
© 2013 Oracle Corporation
Events in detail
63
Event ID
End Time
Start Time
Thread
StackTrace
Payload
Header
Optional
Size
Monday, September 30, 13
Name, Type, Description&
Content Type
© 2013 Oracle Corporation
Event Payload Metadata
66
Monday, September 30, 13
Semantics of a Value
© 2013 Oracle Corporation
Content Type
67
Content Type Displayed asBytes 42 MB
Percentage 42%
Address 0xDEADBEEF
Millis 242 ms
Monday, September 30, 13
Can be multiple Values
© 2013 Oracle Corporation
Content Type
68
Content Type Displayed
Class Class Loader
Name
Modifiers (public,final)
Monday, September 30, 13
© 2013 Oracle Corporation
Event definition in HotSpot
<event id="ThreadSleep" path="java/thread_sleep" label="Java Thread Sleep" has_thread="true" has_stacktrace="true" is_instant="false"> <value type="MILLIS" field="time" label="Sleep Time"/></event>
69
What does it actually mean?path is a identifier for the UI, label is a description of the Eventhas_thread means each event is tied to a Threadhas_stacktrace means get a stack trace for the Eventis_instant decides whether it is instant event or notFinally the payload with a type name and a description.
Monday, September 30, 13
© 2013 Oracle Corporation
Event in C++
EventThreadSleep event;......if (event.should_commit()) { event.set_time(millis); event.commit();}
70
An example of an event in the HotSpot source code.This event repots the time a thread slept.Thread.sleep()
So the event is created and when the thread returns from sleep we check if the event was enabled and should be committed. If so we save the time and commit.
Monday, September 30, 13
© 2013 Oracle Corporation
Or in assembler
281: cmp BYTE [rbp-0x60],0x0 # should commit285: je 300 # jump if zero 287: mov QWORD rbp-0x58],rbx # save time291: lea rdi,[rbp-0x70] # get pointer295: call 0x101b17dc4 # commit
71
This is basically the assembler code generated for the same code (for amd64).Reserve some memory on the stack, initialize it. Check if the event was enabled, otherwise jump away. Save the time, get the pointer and call commit.
The overhead of an event that is disabled is very small. Compare a value to zero and jump away.
Monday, September 30, 13
Promoted to Global buffers-
Global buffers are circular
© 2013 Oracle Corporation
Buffers
73
Global Buffer
Global Buffer
Global BufferGlobal Buffer
Monday, September 30, 13
Self Contained
© 2013 Oracle Corporation
File Format
76
Header Event Records Event Meta Data
Monday, September 30, 13
Enough High-LevelProblems & Solutions
© 2013 Oracle Corporation
Low-level
77
Monday, September 30, 13
Classes are referenced by Events
© 2013 Oracle Corporation
Starting Problem
78
Monday, September 30, 13
org.springframework.security.ui.preauth.PreAuthenticatedGrantedAuthoritiesWebAuthenticationDetails
© 2013 Oracle Corporation
Starting Problem
80
Monday, September 30, 13
Introduce a unique ID
© 2013 Oracle Corporation
Starting Problem
82
class Klass { … u8 _trace_id; …}
17event id
classcount
2422
…17event id
classcount
java.lang.Integer2
…java.lang.Integerjava.lang.Stringjava.lang.Math
java.util.List
Monday, September 30, 13
End of a Recording?-
Classes can unload!
© 2013 Oracle Corporation
Class List
88
Monday, September 30, 13
Lots of classes+ Long class names
=Lots of wasted memory
© 2013 Oracle Corporation
Class List
91
Monday, September 30, 13
Tag referenced classes!
© 2013 Oracle Corporation
Tagged Classes
93
#define CLASS_USED 1
void use_class_id(Klass* const klass) { klass->_trace_id |= CLASS_USED;}
Monday, September 30, 13
Classes come & go.Don’t waste memory
© 2013 Oracle Corporation
Class List
94
Monday, September 30, 13
Writes the Class List,Resets the tags &
clears the Class List
© 2013 Oracle Corporation
Checkpoints
96
class_pool.lookup(242) ➞ java.lang.Integer
method_pool.lookup(314) ➞ java.lang.Math:pow()
Monday, September 30, 13
Classes, Methods, Stack Traces, Threads, Thread Groups, Strings
© 2013 Oracle Corporation
Constant Pool
97
Monday, September 30, 13
Events +Constant Pools +Event Metadata
=Checkpoint
© 2013 Oracle Corporation
Checkpoints - revisit
98
Monday, September 30, 13
© 2013 Oracle Corporation
Checkpoints - revisit
99
eventsconstant pools
meta-data...
checkpoint
Monday, September 30, 13
Contain everything requiredto parse the events prior
to the checkpoint
© 2013 Oracle Corporation
Checkpoints - revisit
100
Monday, September 30, 13
© 2013 Oracle Corporation
More information
‣Whitepaperhttp://www.oracle.com/missioncontrol
‣User Guidehttp://docs.oracle.com/javase/7/docs/technotes/guides/jfr/index.html
‣Forumhttp://forums.oracle.com/community/developer/english/java/java_hotspot_virtual_machine/java_mission_control
101
Monday, September 30, 13