harmony tptp

© 2002 IBM Corporation

Confidential | Date | Other Information, if necessary

Eclipse* TPTP ProjectApache* Harmony Project

© 2008 Intel Corporation; made available under the EPL v1.0 | March 2008

A Harmonious Combo : Experiences Profiling with Eclipse* TPTP

Chris Elford -- Intel CorporationSergey Kuksenko -- Intel Corporation

**

Harmony logo source: http://harmony.apache.org/Harmony logo source: http://harmony.apache.org/

A Harmonious Combo: Experiences Profiling Apache* Harmony with Eclipse* TPTP © 2008 Intel Corporation; made available under the EPL v1.0 2


Outline

Overall project goals Introduction to Apache* Harmony The Eclipse* TPTP profiler First experiences using TPTP Analysis of advanced workload with TPTP Looking to the future



Goals and Methodology of our Experiments Goals

Identify Harmony performance opportunities using TPTP Integrate TPTP into our performance methodology

Approach Start simple…

Verify that we can recognize known performance issues w/ TPTP Understand how opportunities manifest in TPTP Understand current limitations of TPTP and Harmony JVMTI

Fix critical issues along the way

Grow… Move forward looking for new opportunities to optimize Harmony Use TPTP more regularly Suggest TPTP & Harmony JVMTI enhancements for our use case

Most of what we show today is in the “Start Simple” category



Apache* Harmony Overview

Primary Goal of Harmony “a large and healthy community of those interested in runtime

platforms”

Creation of “A compatible, independent implementation of the Java SE 5

JDK under the Apache License v2” “A community-developed modular runtime (VM and class

library) architecture.”

Why? Enables innovation/research and adoption

w/o license restrictions that limit consumption

Source: http://harmony.apache.org/Source: http://harmony.apache.org/



Apache* Harmony Overview (continued)

Why J2SE 5? “Starting with Java SE 5, as that is the first version of Java SE for

which the licensing allows an open source implementation” “Continue with Java SE 6 and any subsequent versions that follow.”

Who all is using Apache* Harmony technology? Since its an open license, we don’t know for sure… but to name a

few http://mail-archives.apache.org/mod_mbox/harmony-dev/200712.mbox/%3c47

[email protected]%3e http://mail-archives.apache.org/mod_mbox/harmony-dev/200711.mbox/%3c80

[email protected]%3e

Harmony is a clean room implementation of J2SE specdistributed under Apache* V2 license

Source: http://harmony.apache.org/faqSource: http://harmony.apache.org/faq



A Brief History of Apache* HarmonyMay 2005: Accepted by the ASF and started in Apache*

IncubatorMay 2006: Harmony has VM and classlib functional to run

Eclipse* 3.0May 2006: JavaOne - demonstrated functional classlib with

Swing/AWTJuly 2006: First JRE Snapshot PublishedOct 2006: Became ASF Top Level ProjectMay 2007: First Milestone Build Released (M1)May 2007: JavaOne – “In Harmony with Eclipse” bundle CDs

distributed…Dec 2007: Fourth Milestone Build Released (M4)Feb 2008: Fifth Milestone Build Released (M5)

Note: We plan to run Eclipse on Harmony in the demo later in talk



Apache* Harmony: What’s Inside? ~2.3 million LOC

~1.6 million Java* code, ~ 0.7 million C/C++ Serious testing and engineering infrastructure Components

API: ~99% JDK5, ~90% JDK6 Project VMs: DRLVM (and others) Tools: javac, javah, javap, jarsigner, keytool, appletviewer

Platform Support Systems with x86 processors supporting Intel® SSE or greater Systems using processors compatible with Intel® EM64T Intel® Itanium®-based systems

Operating System Support Several versions of Windows* and Linux



Apache* Harmony and Eclipse*: Synergy

The Eclipse* platform was the first large application enabled on Harmony back in ’06

Harmony test base includes >20 Eclipse* scenarios Harmony performance has been tuned for Eclipse* platform

Harmony adopted EUT for daily execution Harmony JVMTI has proven to work with TPTP

Harmony adopted key TPTP JVMTI tests for daily execution “In Harmony with Eclipse” bundles updated regularly

http://sourceforge.net/projects/eclipse-harmony/



Eclipse* TPTP Profiling Tool Overview Broadly useful for performance analysis and for gaining a

deeper understanding of a Java* program Consists of the Profiling and Logging Perspective and a

number of graphical and tabular views Visualize program execution and threading behavior;

Pinpoints operations taking most resources; Explore patterns of program behavior

Enables you to test your application's performance early in the programming development cycle for improvements

Option to run application with agent at near full speed (enabled mode) and attach later at certain application phase.

Assorted filtering functionality which can help to localize problem and reduce overhead for large long run applications



Obligatory Eclipse* TPTP Architecture Diagram

Presentation System

Test

Trace

EMF Data Models

Log

StatisticalXMI

Real TimeExport

Runtim

eM

onitor / log

Trace Analysis

And Profiling

Test Creation

and Execution

Artifact

Managem

ent

Eclipse TPTP GUI

Standard Widgets andCore Plug-ins

Reference PerspectivesAnd Workflow

Target System

Data C

ollection

ApplicationD

ata Collection

Interface

InjectionCorrelation

ExecutionEnvironment

Log Collection

Trace Collection

System PerformanceMonitor

JVMPI Monitor

Data Loader

DistributedData Collection

Framework

Agent

Control Interface

Agent

Control Interface

DistributedControl

Framework

Testability Interface

Test Engine

Eclipse Platform

JVMTI Monitor



The New Java* Profiler (JVMTI) vs. JVMPI Java* 5.0 introduced new standards for profiling

Prior to Java* 5.0 (Java* 1.4-) the standards and the interfaces to support profiling (JVMPI) were experimental Java* J2SE 6 eliminated JVMPI interface

The new standard (JVMTI): an innovative solution to profiling; enables precise control over application parts to profile

Eclipse* TPTP has embraced this innovation and the new Java profiler is based on this new standard

In Eclipse* TPTP old Java profiler (JVMPI) still exists Only for backward compatibility with Java 1.4 Because of JVMPI/JVMTI interface differences there are some

differences in behavior between old and new profiler Eclipse* TPTP will eventually remove Old Java* profiler



Eclipse* TPTP Profiler Updates for Ganymede Improve overall usability via assorted fixes

Using feedback from users

Reduce usability diffs between JVMPI and JVMTI… e.g., SSL for secure communications e.g., Reduce overuse of dynamic attach/detach

Enhancements e.g., Better multithread analysis

Improved visualization Contended lock analysis Track more thread states (e.g., join points)

e.g., Efficient binary transfer format e.g., Better Java* 6 support



Using TPTP for Harmony performance analysis: startup optimizations.

Motivation of using TPTP on HarmonyWhen you have 1.6 million lines of Java* code… Team already uses profiling tools extensively

Intel® VTune™ tools heavily usedCorrelation of architectural events to assembly codeVM and JIT code quality analysis and great optimizationMainly use batch mode (execute then perform offline analysis)

Want to also do higher level Java* language centered analysisRich Java* centered analysis of call graphPackage and Class and Method summarizationJava* Thread and Heap analysisIdeally continue with batch mode analysis model

Why TPTP Profiler?Leverage open source to create open sourceMaximize collaboration potential w/ engineer next door



Harmony analysis with TPTP: Startup performance Consider simplest workloads

Hello World Startup phase of a GUI application

Goal Understand detailed end to end behavior Find inefficiencies in class libraries for optimization

Approach Disable default filters in TPTP Collect trace Analyze

Discoveries from our first use of TPTP Even for small apps, data volume can be huge w/o filters Several TPTP usability bugs; fixes targeted for Ganymede Xml4profiling lacks command line tool for tsv/csv dumps



A few words on filtering in TPTP Collection Time Filtering

Pro: Reduce data volume and instrumentation overhead (critical) Con: Data for excluded classes/method can never be seen

Model Import Filtering Pro: Reduce workbench memory footprint (critical on 32bit OSes) Con: Can only see data that is in the model

View Level Filtering Pro: Remove items from being visualized in the workbench Con: Does not reduce data volume or model footprint



Harmony performance analysis w/ TPTP: HWA startup.

5% boost expected from moving key signed jar to bootclasspath.

Sort by number of calls

write(int) is at the top

Sort by base time

java.util.jar.IninManifest:.read() at the top

Callgraph:java.util.jar.InitManifest:.read()->java.io.ByteArrayOutputStream:.write(int)Too much time is spent when reading the manifest of signed jar file...

http://issues.apache.org/jira/browse/HARMONY-5277http://issues.apache.org/jira/browse/HARMONY-5277



See http://sourceforge.net/projects/eio-harmony/

4% boost expected by replacing linear search w/ hashed search in lookup

Harmony perf analysis w/ TPTP: EIOffice startup.

Sort by number of calls

Java.lang.reflect.Method.getName() is at the top

further investigation shows that Method:.getName() is used in linear search for finding methods of class from java.lang.Class.getMethod(String)




More advanced study: SPECjbb* 2005 Analyze more complex, long running, workload

Snapshot performance during workload steady-state We use SPECjbb* 2005 workload for some of our performance tests

Goal Understand application behavior Find inefficiencies in class libraries for optimization

Ideal Approach Disable default filters in TPTP, Collect trace, Analyze

Would prefer to collect trcxml from our batch scripts then do offline analysis

Discoveries from our use of TPTP Can’t get sufficient detail with filtering; w/o filtering; too much data

Pause/resume helps… But doesn’t help allow batch mode execution Hope for richer control with Application mode in Ganymede

Avoid attach mode Instrumentation during run perturbs performance



Performance study: SPECjbb* 2005

spec.org class

3rd party library classes

Overall Execution statistics

•SPECjbb* 2005 source cannot be modified•3rd party library classes hard to influence

Apply filter(view level filter)




Cumulative Time

Candidates for

investigation

Sort by Base Time



Performance study: SPECjbb* 2005First

candidate class Hot

methods

•Next() and nextInt() are rather small.•Typically disappear in JIT inlining

•JIT behavior invisible to TPTP•Class instrumentation impacts behavior

•Message: be careful with getters/setters/etc •TPTP can be set to automatically filter them

•We need to see them so we can double check JIT

•No opportunity for tuning... Move along.




TreeMap iteration is expensive

next() at the top

Look into details



Performance study: SPECjbb* 2005next() is hot

next() invokes get() pretty much

each time


Problem!• Single generalized iterator

impl. (e.g., KeyIterator, ValueIterator, EntryIterator)

• Ignores common cases for which optimization is possible

Also… •Creates excess “getter”

objects.• Inlining complicated

due to interface call5% boost from implementing three specialized

iterators for common use cases




Next hot classes:

java.math.*Choose hot method for

details



Performance study: SPECjbb* 2005 – java.math.* (~10%)

Each BigDecimal.add() invokes BigInteger.add()

A BigDecimal is a BigInteger plus a scaling factor

Do we really need to create separate BigInteger object for each BigDecimal value?

• Let’s incorporate small values (<19 digits) directly into BigDecimal.

Result: More then a 30% boost!!!

Short Quiz:java.math.* took ~10%?How did we get more then 30%?

Only 1 object (BigDecimal) is created •Instead of 3 objects (BigDecimal, BigInteger, int[])•Less GC stress!




What to do next to analyze Harmony with TPTP?So now that we’ve tried TPTP, what is next? Use Eclipse TPTP profiler often to improve Harmony

Work closely with Eclipse* TPTP developers So they better understand how we use TPTP

Use heap profiler Harmony’s memory footprint is fairly large Target smaller heaps (e.g., for J2SE on smaller devices) Already touch tested it

Use thread profiler Particularly excited about optimizing multithreaded Java for

multicore platforms Already touch tested it



Summarizing wish lists for TPTP and Harmony

Features we would like to see in TPTP in future Better handling of huge datasets (multi-GB) Simpler and richer batch (scripted) collection Command line data reduction utility (offline analysis) Interface for filtering “easily” inlinable getter/setter methods at

collection time

Features we would like to see in Harmony in future Fuller JVMTI implementation

e.g., RedefineClasses - needed for TPTP dynamic attach/detach Better memory footprint on large Eclipse* scenarios Further tuned Eclipse* workbench startup time Java* J2SE6 support



Conclusion

Our perspective Apache* Harmony making great progress Eclipse* TPTP allows great Java* code analysis Both projects have opportunities for further enhancement

Look forward to future collaboration between Harmony and TPTP

What we would like from you

Go try Harmony (w/ Eclipse* platform) Tell us about your experience (and feel free to join)

Go try Eclipse* TPTP profiler Tell us about your experience (and feel free to join)



TPTP Resources (profiling and beyond) Learn and Try

http://www.eclipse.org/tptp/home/documents/conferences/eclipseCon2007http://www.eclipse.org/tptp/home/documents/conferences/eclipseCon2007/%283669%29%20Profiling%20Java%20applications%20using%20Eclipse%20TPTP%20v1_1.htm

Webs and Wikishttp://eclipse.org/TPTPhttp://wiki.eclipse.org/TPTP

Downloads and Updateshttp://www.eclipse.org/tptp/home/downloadshttp://www.eclipse.org/tptp/home/downloads/updateManager.php

News and Mailhttp://www.eclipse.org/tptp/home/project_info/general/mailnews.php

Use and Participatehttp://wiki.eclipse.org/TPTP_User_Experiences_Profilinghttp://www.eclipse.org/tptp/home/project_info/general



Harmony Resources Learn and Try

http://harmony.apache.org/hdk.htmlhttp://harmony.apache.org/subcomponents/drlvm/index.html

Webs and Wikishttp://harmony.apache.org/http://wiki.apache.org/harmony/

Downloads and Updateshttp://harmony.apache.org/download.cgihttp://sourceforge.net/projects/eclipse-harmony/

Use and Participatehttp://harmony.apache.org/quickhelp_contributors.html



Intel, the Intel logo, Itanium and VTune are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

* Other names and brands may be claimed as the property of others.



Questions?



BACKUP



Eclipse* TPTP -- The New Java* Profiler

A set of libraries that attach to a JVM for recording Java application's behavior

An extensible framework, consisting of a core runtime component (Martini), an agent managed by the Agent Controller (JPIAgent), and a set of data collection libraries built on top of the Martini runtime.

Can be used to identify performance details such as classes or methods responsible for execution bottlenecks, analyze application heap to find memory leaks and visualize threading behavior.

Output in the form of XML fragments (XML4Profiling) Can be launched from the Eclipse IDE or as a standalone program using

Java command-line options Applications under test can reside in Eclipse workspace, binaries on file

system, or hosted in a J2EE application server.



Apache* Harmony Status: Overall

API completeness: ~99% Remaining 1% is in Swing as well as spread over all code

Non conformant – Sun* is silent on TCK issue. Compatibility is proven by a number of free and commercial apps

running on Harmony VM completeness: ~100% (according to the spec)

Interpreter, 2 JITs, DPGO Optimized JIT is based on StarJIT – initially tuned to show best

performance on Intel® Itanium® processors – has a good large enough set of HLOs

Parallel, generational GC with innovative algorithms. Two commercial-quality VMs are working with Harmony classlibs:

IBM* J9* BEA* JRockit*

Performance is good on server benchmarks, client still needs some attention.

Large set of test suites: Unit, VTS (conformance), reliability, stress, performance, etc.



How we collected a snapshot w/o filters

1. Configure profiling to save to a file Destination tab in workbench

2. Configure aggregated stats with 3600 second refresh Avoid auto refresh… Only manual

3. Start the run and immediately pause collection Pause button… Could not find a way to start paused…

4. Wait until steady state reached5. Resume collection6. Wait a few minutes7. Pause collection8. Wait for data to come across9. Terminate

Do not save10. Restart Eclipse

Ensures maximum memory available for file import11. Import from where you saved file in (1).



Heap profile: a first try



Thread profile: a first try

harmony tptp

Documents