performance tuning apache tomcat steve heckler, president accelebrate [email protected]
TRANSCRIPT
Performance TuningApache Tomcat
Steve Heckler, President
Accelebrate
http://www.accelebrate.com
2
What We’ll Cover• Removing unneeded applications
• Tuning and monitoring the JVM
– This section “borrows” some from Sun’s official documentation
• Tuning and monitoring connectors
• Compiling native connectors
• Tuning and monitoring database connection pools
• Turning off JSP development mode
• Reducing logging
• Precompiling JSPs and caching output
3
Tomcat Version in Use
• We’ll use Tomcat 6, but the majority of strategies shown are applicable to earlier versions
4
Tomcat Versions (see http://tomcat.apache.org for details)
General Version
Latest Version JDK Servlet API JSP API
3.3.x 3.3.2 1.2+ 2.2 1.1
4.1.x 4.1.36 1.2+ 2.3 1.2
5.0.x 5.0.28 1.4+(1.3)
2.4 2.0
5.5.x 5.5.25 1.5+ (1.4)
2.4 2.0
6.0.x 6.0.14 1.5+ 2.5 2.1
Word of Advice #1:Use a Recent Java SE version
• Java 1.5 sizes heap generations much more efficiently than 1.4 and earlier
• Java 1.6 has an option to perform garbage collections in parallel (more about this later)
5
Remove Unneeded Applications
• At your discretion, you can remove every installed application that ships “out of the box” in Tomcat
• This will save you startup time, as well as memory used by preloaded servlets
• ROOT should definitely be replaced
• Only keep manager (and in 5.5 and later, host-manager) if you need them
6
Tomcat Monitoring
• Tomcat is difficult to monitor prior to Java 1.5 and later builds of Tomcat 4.1.x
• Java 1.5 and later support monitoring the JVM using JConsole and the jstat command line tool (both included with the JDK)
• Tomcat 5.0 and later have especially good JMX MBeans support
7
What are JMX and MBeans
• JMX: Java Management Extensions, a standard way of managing Java applications
• MBeans: Management beans. An application can provide management beans that enable you to interact with and configure the applications. MBeans have:– Attributes that you can get or set
– Operations that you can invoke
• MBeans can be interacted with programmatically or via JConsole
8
Enabling JMX monitoring(via setenv.sh)
#!/bin/shif [ "$1" = "start" ] ; then CATALINA_OPTS="$CATALINA_OPTS \ -Dcom.sun.management.jmxremote" # -Dcom.sun.management.jmxremote.port=9086 \ # -Dcom.sun.management.jmxremote.ssl=false \ # -Dcom.sun.management.jmxremote.authenticate=false" echo $0: CATALINA_OPTS = "$CATALINA_OPTS"fi
9
Enabling JMX Monitoring (Windows)
• In a production environment, you can set these in CATALINA_OPTS by editing the service.bat before registering the service
• Alternatively, edit the registry keys directly after the service is registered
10
JMX Monitoring: Key Decisions
• Only specify a port if you want to allow remote access (potentially a security hole)
• Consider configuring authentication and/or SSL if you do open a port– http://java.sun.com/j2se/1.5.0/docs/guide/
management/agent.html has details on this
11
Once JMX is enabled
• You can run jconsole from the command line of any computer with JDK 1.5 or later installed
• Accessing your JVM remotely will require that a port be opened
• jstat can be run locally to monitor your JVM (dumps output to STDOUT)
12
13
Introduction to JavaGarbage Collection
• Java memory utilization and garbage collection are two of the most critical issues in Tomcat performance tuning.
• Garbage collection is the process whereby memory is reclaimed from the application
• Java tends to shield developers from control of garbage collection, but not from the consequences
14
Introduction to JavaGarbage Collection
• In Java, objects become eligible for garbage collection when they no longer have any references pointing to them
• The story of how this occurs is rather complicated…
15
GC Scalability(graphic shown at http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html)
16
GC Scalability
• Demonstrates that an application that spends 10% of its time in garbage collection can lose 75% of its throughput when scaled out to 32 processors
17
Heap versus Non-Heap Memory
• JVM manages 2 kinds of memory: heap and non-heap:– Heap memory is the runtime data area from
which the JVM allocates memory for all class instances and arrays.
– The heap may be of a fixed or variable size.– The garbage collector is an automatic memory
management system that reclaims heap memory for objects.
18
Heap versus Non-Heap Memory
– Non-heap memory includes a method area shared among all threads and memory required for the internal processing or optimization for the JVM.
– It stores per-class structures such as a runtime constant pool, field and method data, and the code for methods and constructors.
19
Heap versus Non-Heap Memory
– A JVM implementation may require memory for internal processing or optimization which also belongs to non-heap memory.
– For example, the JIT compiler requires memory for storing the native machine code translated from the JVM code for high performance.
20
Garbage Collection
• Garbage collection (GC) is how the JVM frees memory occupied by objects that are no longer referenced.
• It is common to think of objects that have active references as being "alive" and un-referenced (or unreachable) objects as "dead.“
• Garbage collection is the process of releasing memory used by the dead objects. The algorithms and parameters used by GC can have dramatic effects on performance
21
Generational Garbage Collection
• In practice, most programs create:– many objects that have short lives (for example,
iterators and local variables). – some objects that have very long lifetimes (for
example, high level persistent objects)
22
Generational Garbage Collection
• GC divides memory into several generations, and assigns each a memory pool.
• When a generation uses up its allotted memory, the VM performs a partial garbage collection (also called a minor collection) on that memory pool to reclaim memory used by dead objects.
• This partial GC is usually much faster than a full GC.
23
Generational Garbage Collection
• Generations:– young generation (“the nursery”)
• eden space– Most objects initially assigned here (and die here)
• two survivor spaces– Objects that survive a minor GC in eden space are moved here
– old generation• tenured space
– Objects that survive long enough in the survivor spaces– When tenured space fills up, full GC occurs (often slow and involves all
live objects)– permanent generation
• holds all the reflective data of the virtual machine itself, such as class and method objects
24
Generational Garbage Collection
• If the garbage collector has become a bottleneck, you may be able to improve performance by customizing the generation sizes.
• http://java.sun.com/docs/hotspot/gc/index.html details how to customize these sizes.– Java 6 version at
http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html
• Customizing the sizes is less necessary in Java 1.5 and later
25
Tuning the Total Heap
• Total available memory is the most important knob affecting GC performance
• By default, the JVM grows or shrinks the heap at each collection to try to keep the proportion of free space to living objects at each collection within a specific range
• This target range is set as a percentage by the parameters -XX:MinHeapFreeRatio=<minimum> and -XX:MaxHeapFreeRatio=<maximum>, and the total size is bounded below by -Xms and above by -Xmx
26
Tuning the Total Heap
• Some notes:– Unless you have problems with pauses, try granting as
much memory as possible to the JVM. The default size (64MB on a 32-bit OS) is often too small.
– Setting -Xms and -Xmx to the same value increases predictability by removing the most important sizing decision from the JVM.
– Be sure to increase heap size as you add cores or processors
– 32-bit OSes cap the heap size at between 1.5 and 2.5GB
Tuning and Monitoring Connectors
• Tomcat supports connectors for http, https, and ajp• Suggestions:
– Set enableLookups to false
– Make sure maxThreads and acceptCount are set sufficiently high (but not so high that you’re accepting more traffic than your Tomcat instance can handle)
– compression trades off bandwidth and processing time
27
Alternatives to the ClassicJava Blocking Connector
• NIO– New input/output– Supported starting in Tomcat 6– Provides access to low-level I/O operations of modern operating
systems, including multiplexed, non-blocking I/O and polling
• APR connector (Apache Portable Runtime)– Tomcat 5.5.15 and later (reliably)– Uses OpenSSL– Native code and thus faster
• Bottom of http://tomcat.apache.org/tomcat-6.0-doc/config/http.html has a good comparison
28
Building the Native Connectors
• Dependencies– OpenSSL (you need the source) - 0.9.8a or later
– APR (on Red Hat and variants, apr-devel RPM is sufficient) - 2.2 or later
• Steps:– Locate or download APR
– Download OpenSSL (you do need the source)
– Unpack the native connector
29
Building the Native Connectors
• Steps (continued)– ./configure --with-ssl=[path to extracted SSL] --
with-apr=[path to apr-1-config, possibly /usr/bin/apr-1-config]
– make– make install– Update setenv.sh to use the built library CATALINA_OPTS="$CATALINA_OPTS -Djava.library.path=/usr/local/apr/lib"
30
Building the Native Connectors
• Steps (continued)– configure the APR connector(s) in server.xml– Update protocol attributes:
• HTTP: org.apache.coyote.http11.Http11AprProtocol
• AJP: org.apache.coyote.ajp.AjpAprProtocol
– Optionally, configure SSL and set up the APR https connector
31
Building the Native Connectors
• Sample APR SSL connector
<Connector protocol="org.apache.coyote.http11.Http11AprProtocol" port="8443" minSpareThreads="5" maxSpareThreads="75" enableLookups="true" disableUploadTimeout="true" acceptCount="100" maxThreads="200" scheme="https" secure="true" SSLEnabled="true" SSLCertificateFile="/usr/local/ssl/server.crt" SSLCertificateKeyFile="/usr/local/ssl/server.pem" clientAuth="false" sslProtocol="TLS"/>
32
Monitoring DatabaseConnection Pools
• Can be done via /Catalina/DataSource in the MBeans tree within JConsole
• Parameters for your pool are shown at http://commons.apache.org/dbcp/configuration.html
• Be sure to make your pool large enough for the traffic you anticipate
• Be sure to time out requests for connections• Consider removing abandoned connections
33
Other Suggestions
• Turn off JSP development mode in production by setting the development parameter of the jsp servlet to false– Keeps JSPs from being checked for modification
• Scale back access and error logging to just what’s needed
• Consider precompiling your JSPs – see http://tomcat.apache.org/tomcat-6.0-doc/jasper-howto.html#Web%20Application%20Compilation
34
35
In Closing
• Thank you for joining me!
• Please email me at [email protected] with questions
• Good luck with tuning Tomcat!