monitoring oracle soa suite - ukoug tech15 2015
TRANSCRIPT
![Page 1: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/1.jpg)
8th December 2015 UKOUG Tech15Matt Brasier
Monitoring Oracle SOA Suite
![Page 2: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/2.jpg)
About me
• Head of Consulting at C2B2• 14 years WebLogic experience• 12 in a consultancy role• Author– Oracle SOA Suite 11g Performance Tuning
Cookbook
![Page 3: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/3.jpg)
About C2B2
• The leading independent middleware experts• Middleware professional services– Consultancy– On-site Support
• Independent– Oracle partner– Red hat partner
![Page 4: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/4.jpg)
Agenda
• Intro• Why Monitor?• How to Monitor?• What to Monitor?
![Page 5: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/5.jpg)
Introduction
![Page 6: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/6.jpg)
Oracle SOA Suite
• Evolved rather than designed– BPMN– BPEL– Mediator– Rules– Workflow– BAM/B2B/Event Processor
• This has an impact on the ability to monitor it
![Page 7: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/7.jpg)
Oracle SOA Suite
• Runs on a stack– Hardware– OS– JVM– WebLogic– SOA Suite
• A skyscraper not a monolith
![Page 8: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/8.jpg)
Monitoring
• Monitoring– Capturing of metrics– Visualising of metric trends
• Alerting– Generating a notification when a condition is met• Out of bounds metric
![Page 9: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/9.jpg)
The Java Monitoring world
• Still developing– Going backwards?
• Lots of tools• Monitoring for deployments of all sizes• Monitoring frameworks vs Alerting
frameworks• Java focussed frameworks vs frameworks that
can monitor java
![Page 10: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/10.jpg)
Why Monitor?
![Page 11: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/11.jpg)
Six reasons to monitor SOA Suite
• Detect problems early• Capacity planning• Resolve problems faster• Understand your system better• Understand your business• Save money
![Page 12: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/12.jpg)
Detect problems early
• Locate areas of resource contention• Identify unusual workloads• Identify failed components or services• Detect problems in other systems– Middleware is often the best place to start
• Alerting is key– Alerts to the right people at the right time
![Page 13: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/13.jpg)
Capacity planning
• Detect trends in usage• Understand how changes in use will affect
capacity• Increase capacity before it causes a problem– Hardware and upgrade lead times
![Page 14: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/14.jpg)
Resolve problems faster
• Post incident analysis• What resources were at their limits?• What was the system doing before it failed?• What were other systems doing at the time?
• What can we do to alert before failure next time?
![Page 15: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/15.jpg)
Understand your system better
• What does a normal day look like on the system?– Resource usage– Use case load
• Which resources are key to system operation?• What improvements can you make?
![Page 16: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/16.jpg)
Understand your business
• Classic MI– How do “transactions” flow through your
systems?– When are your busiest periods?– When are your quiet periods?– How much impact did the latest advertising
campaign have?
![Page 17: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/17.jpg)
Save money
• The bottom line– Less outages– Faster resolutions– Less repeated failures– Target capacity correctly– More knowledgeable and better prepared
administrators
![Page 18: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/18.jpg)
How to Monitor?
![Page 19: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/19.jpg)
Ways to monitor Oracle SOA Suite
• Manual monitoring• Scripts• Monitoring tools• Log scraping tools• Alerting
![Page 20: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/20.jpg)
Manual Monitoring
• Someone sits and looks at a console– Watching log files– Reviewing output from basic tools• jVisualVM• jstat• JRMC• DMS
![Page 21: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/21.jpg)
Manual monitoring tools
• JVisualVM– Graphical monitoring for Hotspot– Plugin based– Monitors key attributes
• JRockit Mission Control– Graphical monitoring for JRockit– Similar to JVisualVM– Flight recorder
![Page 22: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/22.jpg)
Manual monitoring tools
• JStat– Command line output– Memory– Garbage collection– Compilation
![Page 23: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/23.jpg)
Manual monitoring tools
• DMS– Oracle store of monitoring information– Stored in the database– DMSSpy servlet to view the data• Or interrogate the database
![Page 24: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/24.jpg)
Scripts
• Collect data from manual tools and store it somewhere using a script– WLST– jstat
• Analyse data in tools such as excel when required
![Page 25: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/25.jpg)
Scripts
• Provide a diagnostic script to second line support– Before restarting a “stuck” server, run this script– Capture metrics before they are lost– Better than nothing• Not as good as a real monitoring tool
![Page 26: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/26.jpg)
Monitoring Tools
• Usually server/agent based– Install agent on the host to be monitored
• Wide range of tools– Oracle Enterprise Manager Cloud Control– Nagios– ManageEngine– HP OpenView
![Page 27: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/27.jpg)
Monitoring Tools
• Specific vs Generic– Specific tools often provide more out of the box– Generic tools often need some configuration
• Auto discovery vs manual configuration• Where do they store the data?• How much information do they provide– OS– Hardware
![Page 28: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/28.jpg)
Which monitoring tool?
• What infrastructure is in place already?– Is OEM already being used to monitor databases?– Is Nagios already in use by operating system
teams?– Who controls the monitoring tools?
![Page 29: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/29.jpg)
Which monitoring tools?
• No really great tools for SOA Suite• Rough order of (personal) preference– Oracle Enterprise Manager Cloud Control 12c– Manage Engine– HP Openview– Nagios
![Page 30: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/30.jpg)
Log Scraping Tools
• Splunk is the best known example• Send metrics to a log file• Log scraping tool reads and parses the
metrics• Log scraping server integrates metrics
together and displays them
![Page 31: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/31.jpg)
Monitoring best practices
• Capture key metrics (see later)• Don’t alert on everything you capture• Consider data retention periods• Be open with access to monitoring data• Monitoring configuration owned by support
teams who use it
![Page 32: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/32.jpg)
Alerting best practices
• Who is in a position to do something about the alert
• The people who have to deal with it should own the alert– Thresholds– Enabling/disabling– Damping
![Page 33: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/33.jpg)
Enterprise Monitoring
• Architectural options– Generic enterprise monitoring tool– Specialist monitoring tool that sends alerts up
stream– Multiple monitoring tools• Network• Hardware• OS• Middleware
![Page 34: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/34.jpg)
Enterprise monitoring
• Best approach seems to be tiered tools– SOA support team own their own tool– SNMP to push alerts up to enterprise monitoring
solution
![Page 35: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/35.jpg)
What to Monitor?
![Page 36: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/36.jpg)
What to monitor?
• Hardware• OS• JVM• WebLogic• SOA Suite• Log files
* Indicates alert candidate
![Page 37: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/37.jpg)
Hardware
• Disk usage*• Network IO*• CPU Usage*• Memory Usage
![Page 38: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/38.jpg)
Operating System
• File handles*• User CPU vs System CPU• Modification dates on key files*• Boot time *
![Page 39: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/39.jpg)
Java Virtual Machine
• Garbage Collection*• Memory Usage– Old– New– Perm…
• Threads
![Page 40: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/40.jpg)
WebLogic
• Datasource connection pool size*• JMS Queue/Topic depth*• JMS Consumer count*• Stuck threads*• Work managers
![Page 41: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/41.jpg)
SOA Suite
• Composite response times
![Page 42: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/42.jpg)
Log Files
• Key exception or error types– java.io.IOException– oracle.jdbc.– java.lang.OutOfMemoryError– .– .– .
![Page 43: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/43.jpg)
What Next?
![Page 44: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/44.jpg)
What Next?
• Write a monitoring policy• Review your existing monitoring against it• Update your monitoring• Improve your availability• Resolve issues faster
![Page 45: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/45.jpg)
What Next?
• Follow C2B2 on twitter– @c2b2consulting
• Follow me on twitter– @mbrasier
• View our SOA Suite resources page– http://info.c2b2.co.uk/soa-suite-resources
![Page 46: Monitoring Oracle SOA Suite - UKOUG Tech15 2015](https://reader036.vdocuments.mx/reader036/viewer/2022062412/587c19551a28abb5068b4dcb/html5/thumbnails/46.jpg)
Questions?