Download - DS Monitoring
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 1/35
Data Services Monitoring with
Solution Manager 7.1Heinz Wolf
2014
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 2/35
© 2011 SAP AG. All rights reserved. 2
The process behind technical operations…
Monitor
Proactive real-time
monitoring
Optimize
Optimize excellence of
technical operations
Analyze
Lower mean time to
problem resolution
Notify
Reactive handling of
critical events
Report
Prove value to business
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 3/35
© 2011 SAP AG. All rights reserved. 3
Technical Monitoring and Alerting Capabilities in Detail
Unified Alert Inbox• Central access point for all alerts coming from the different monitoring scenarios• Integration of Incident & Notification Management, Root Cause Analysis and collaboration features
System Monitoring• Status overview for technical systems, instances, databases and hosts
• Drill down to single metrics and events, Jump in to metric reporting and landscape information
End User Experience Monitoring• Measurement of availability and response times from an end user perspective
• Deep integration in E2E Trace Analysis for Root Cause Analysis
Process Integration Monitoring• Central entry point for SAP Process Integration specific monitoring for complete PI domains
• Contains central monitors for PI components, PI channels and Message flows
Business Intelligence Monitoring• Central monitoring for SAP Business Intelligence solutions based on SAP BW and BOE XI
• Monitoring of SAP Business Warehouse process chains and Business Objects specific jobs
Connection Monitoring• Active Monitoring of RFC and HTTP connections between SAP Systems
I n t e g r a t i o n o f a l l s c e n
a r i o s i n
E n d - t o - E n d
M o n i t o r i n g a
n d A l e r t i n g
I n f r a s t r u c t u r e
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 4/35
© 2011 SAP AG. All rights reserved. 4
Data Services Monitoring(Solution Manager 7.10 >= SP 05)
System Monitoring available for Data Services 4.0
• Per Job server memory and CPU
utilization are monitored
• Job specific metrics (job selection can be
configured by customer)
• Job runtime
• Job finished status
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 5/35
© 2011 SAP AG. All rights reserved. 5
Data Services Monitoring(Solution Manager 7.10 >= SP 05)
System Monitoring available for Data Services 4.1
• Availability monitoring for each Job
Server and Access Server
• Per Job server memory and CPU
utilization are monitored
• Job specific metrics (job selection can be
configured by customer)
• Job runtime
• Job finished status
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 6/35
© 2011 SAP AG. All rights reserved. 6
BI Platform Monitoring (Solution Manager 7.10)
• Data Services 4.x requires a BI Platform system (e.g. for user management)
• BI platform system is a separate technical system in Solution Manager and to be setupseparately
• Most important compoent for Data Services is CMS (Central Management Server)
• CMS Availability and Performance metrics
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 7/35© 2011 SAP AG. All rights reserved. 7
Technical Monitoring and Alerting Capabilities in Detail
Unified Alert Inbox
• Central access point for all alerts coming from the different monitoring scenarios• Integration of Incident & Notification Management, Root Cause Analysis and collaboration features
System Monitoring• Status overview for technical systems, instances, databases and hosts
• Drill down to single metrics and events, Jump in to metric reporting and landscape information
End User Experience Monitoring• Measurement of availability and response times from an end user perspective
• Deep integration in E2E Trace Analysis for Root Cause Analysis
Process Integration Monitoring• Central entry point for SAP Process Integration specific monitoring for complete PI domains
• Contains central monitors for PI components, PI channels and Message flows
Business Intelligence Monitoring• Central monitoring for SAP Business Intelligence solutions based on SAP BW and BOE XI
• Monitoring of SAP Business Warehouse process chains and Business Objects specific jobs
Connection Monitoring• Active Monitoring of RFC and HTTP connections between SAP Systems
I n t e g r a t i o n o f a l l s c e n
a r i o s i n
E n d - t o - E n d
M o n i t o r i n g a
n d A l e r t i n g
I n f r a s t r u c t u r e
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 8/35© 2011 SAP AG. All rights reserved. 8
Introduction - Monitoring for Business Intelligence
SAP Solution Manager
BI Overview Monitor
System Monitor
Central system status overview for all
technical components involved in SAP
Business Intelligence Solution
Capability to monitor cross-systemSAP BW process chains and single process
chain steps
Integration of Business Intelligence specific
alerts in Alert Inbox including Notification
Management, Incident Management, Task
Assignment and forwarding to 3rd party
Central monitoring of Business Objects/Data
Services specific jobs and correlation tosystem specific metrics
Central monitoring of SAP BW queries and
templates
BI Detail Monitors
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 9/35
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 10/35© 2011 SAP AG. All rights reserved. 10
BI Monitoring – BI Overview Monitor (graphical view)
The BI Overview Monitor shows at a glance the summary of the current status and the openalerts of all BI components that are included in your technical scenario.
Summary of alerts and status worst case of all systems in the BO Web Application Layer, BO
Server Layer and BW System Layer
Alert (summary)
Technical instances status (worst case)
BI monitored objects status (summary)
System + host level status (worst case)
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 11/35© 2011 SAP AG. All rights reserved. 11
BI Overview Monitor with Data Services (Example)
Summary status of monitored DS Jobs
Summary status of monitored DS System(s)
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 12/35© 2011 SAP AG. All rights reserved. 12
BI Monitoring Configuration – BO Job Monitoring
Data Services 4.0 jobs can also be scheduled via BO job scheduler
BO jobs can be monitored via a specialized application within BI monitoring (as of SolMan 7.1 SP3)
Switch on/off
metric
data collection
Switch on/off
alerting
Switch on/off
monitoring of job
return status
(default = on).
Unsuccessful job
execution willresult in red alert.
Only finished jobs
are taken into
account.
Expected job start
time (UTC).
Not started on time
thresholds
are measured
against this referencetimestamp.
Expected time
window (UTC) for
the job to start and
finish within.
Out of time
window thresholds
are meassuredagainst this
reference time
frame.
Thresholds in
minutes for job
start delay. No job
start event found
within „Not started
on time“timestamp +
threshold raises
alert.
Thresholds in
minutes for job
started too early
or finished too
lateThresholds in
minutes for jobrun duration in
minutes
Leaving yellow and red thresholds blank disables the related metric to be monitored
(e.g. start delay will not be monitored if all „Not started on time...“ fields are empty.
Inconsistent threshold values or invalid time settings will not be accepted. Details can be
found in the log section.
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 13/35© 2011 SAP AG. All rights reserved. 13
BO Job Monitoring within „BI Monitoring Scenario“
• Monitoring UI shows all Job metrics for a selected scenario
• Full integration into Alert inbox / Notification / Incident handling
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 14/35© 2011 SAP AG. All rights reserved. 14
BI Monitoring Configuration – DS Job Monitoring
• Preconditions are: DS >= 4.1 and Solution Manager >= 7.1 SP06
Switch on/off
metric
data collection
Switch on/off
alerting
Switch on/off
monitoring of job
return status
(default = on).
Unsuccessful job
execution willresult in red alert.
Only finished jobs
are taken into
account.
Expected job start
time (UTC).
Not started on time
thresholds
are measured
against this referencetimestamp.
Expected time
window (UTC) for
the job to start and
finish within.
Out of time
window thresholds
are meassured
against this
reference time
frame.
Thresholds in
minutes for job
start delay. No job
start event found
within „Not started
on time“timestamp +
threshold raises
alert.
Thresholds in
minutes for jobstarted too early
or finished too
late
Thresholds in
minutes for job
run duration in
minutes
Leaving yellow and red thresholds blank disables the related metric to be monitored
(e.g. start delay will not be monitored if all „Not started on time...“ fields are empty.
Inconsistent threshold values or invalid time settings will not be accepted. Details can
be found in the log section.
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 15/35© 2011 SAP AG. All rights reserved. 15
DS Job Monitoring within „BI Monitoring Scenario“
• Monitoring UI shows all Job metrics for a selected scenario
• Full integration into Alert inbox / Notification / Incident handling
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 16/35© 2011 SAP AG. All rights reserved. 16
DS Job Monitoring with „Job Monitoring Scenario“
• Preconditions are: DS >= 4.1 and Solution Manager >= 7.1 SP12
• Additional metric “Rows Read” available
• Interactive Reporting integration available
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 17/35© 2011 SAP AG. All rights reserved. 17
The process behind technical operations…
Monitor
Proactive real-time
monitoring
Optimize
Optimize excellence of
technical operations
Analyze
Lower mean time to
problem resolution
Notify
Reactive handling of
critical events
Report
Prove value to business
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 18/35
© 2011 SAP AG. All rights reserved. 18
E2E Root Cause Analysis with Solution Manager 7.1
End-To-End Wor kload Analys is• General performance overview for heterogeneous landscape
• Review most important KPI’s cross all technologies and drill down to product
specific workload KPI’s
End-To-End Change Analysis
• Statistical change data cross all technologies based on daily configurationsnapshots
• Compare configurations between systems and drill down to change reporting
for a detailed change history
End-To-End Exc ept ion Analysis
• Statistical exception data cross all technologies for exception trend analysisor review exception after changes
• Jump to component specific exception analysis (ST22, NWA,…)
End-To-End Trace Analysis
• Single user request tracing in a complex system landscapes
• Identify the problem causing component (performance and functional) and jump-into detailed component specific trace analysis (SQL, ABAP, J2EE trace,…)
System, Host & Database Analysis
• Central, safe and remote access to file system, OS and DB
• Links to read-only monitoring and administration tools like Wily Introscope
for performance analysis and monitoring
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 19/35
© 2011 SAP AG. All rights reserved. 19
Root Cause Analysys for Data ServicesE2E Change Analysis
Overview on
number of
configuration
changes
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 20/35
© 2011 SAP AG. All rights reserved. 20
Root Cause Analysys for Data ServicesE2E Change Analysis
• Drilldown into
changes per
„config store“
• Drilldown into
single changes including
before/after values
• Additionally comparision of two
systems or two timestamps for
one system si possible
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 21/35
© 2011 SAP AG. All rights reserved. 21
Root Cause Analysys for Data ServicesE2E Workload Analysis
• Overview with DS Jobs and Data Flow statistics• Here combined with BI Platform statistics
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 22/35
© 2011 SAP AG. All rights reserved. 22
Root Cause Analysys for Data ServicesE2E Workload Analysis
Execution times per Job and
Data Flow
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 23/35
© 2011 SAP AG. All rights reserved. 23
Root Cause Analysys for Data ServicesE2E Workload Analysis
Memory and CPU consumption per DS job server
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 24/35
© 2011 SAP AG. All rights reserved. 24
Root Cause Analysys for Data ServicesE2E Workload Analysis
Memory and CPU consumption per DS job
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 25/35
© 2011 SAP AG. All rights reserved. 25
Root Cause Analysys for Data ServicesE2E Workload Analysis
Memory and CPU consumption per DS data flow
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 26/35
© 2011 SAP AG. All rights reserved. 26
Root Cause Analysys for Data ServicesE2E Workload Analysis
• Statistics per result status /
execution time per DS job
as well as per Data Flow
• Additionally row count statistics per
data flow
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 27/35
© 2011 SAP AG. All rights reserved. 27
Root Cause Analysys for Data ServicesE2E Workload Analysis
• Host level metrics for memory and
CPU load on the host
• Process level monitoring for all DS
or BI platform related OS
processes
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 28/35
© 2011 SAP AG. All rights reserved. 28
Root Cause Analysys for Data ServicesIntroscope Dashboards
• Data Services is
included
within the BOE 4.x
dashboard
• Data Services 4.0 is
also valid for >4.0
(will be removed)
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 29/35
© 2011 SAP AG. All rights reserved. 29
Root Cause Analysys for Data ServicesIntroscope Dashboards
• Overview on CPU and
Memory utilization
caused by all DS job
servers together
• No metrics are reported
when no job is executed
• Host elapsed time shows
Job run times
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 30/35
© 2011 SAP AG. All rights reserved. 30
Root Cause Analysys for Data ServicesIntroscope Dashboards
• CPU and Memory
utilization per DS job
servers (each Job Server
produces a separateline) – in this example
we had only one
• The Job server name
can be seen in the metric
details window
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 31/35
© 2011 SAP AG. All rights reserved. 31
Root Cause Analysys for Data ServicesIntroscope Dashboards
• CPU utilization per job
and data flow (job / data
flow name in detail
window)
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 32/35
© 2011 SAP AG. All rights reserved. 32
Root Cause Analysys for Data ServicesIntroscope Dashboards
• Elapsed times per job
and data flow
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 33/35
© 2011 SAP AG. All rights reserved. 33
SAP Solution Manager Roadmap
Solution Manager Content for Data Services
E2E Root Cause Analysis
SM7.1 SP02 Data Services >=4.0
System Monitoring
SM7.1 SP05 Data Services >=4.0
System Monitoring with DS Availability (DS 4.1)
Integration into BI monitoring scenario
SM7.1 SP06 Data Services 4.0/4.1
1
2
3
Specific DS job Monitoring application in BI
monitoring
SM7.1 SP07 Data Services 4.1/4.24
Specific DS job Monitoring application in Job
monitoring (+records metric, +reporting)
SM7.1 SP12 Data Services 4.1/4.25
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 34/35
© 2011 SAP AG. All rights reserved. 34
Further Documentation
• Best Practice Document for Root Cause Analysis for Data Services
Best Practice Document in SAP Service Marketplace
7/21/2019 DS Monitoring
http://slidepdf.com/reader/full/ds-monitoring 35/35
No part of this publication may be reproduced or transmitted in any form or for any purposewithout the express permission of SAP AG. The information contained herein may bechanged without prior notice.
Some software products marketed by SAP AG and its distributors contain proprietarysoftware components of other software vendors.
Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of MicrosoftCorporation.
IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x,System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries, zSeries, eServer,z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server,PowerVM, Power Architecture, POWER6+, POWER6, POWER5+, POWER5, POWER,OpenPower, PowerPC, BatchPipes, BladeCenter, System Storage, GPFS, HACMP,
RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX,Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registeredtrademarks of IBM Corporation.
Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks orregistered trademarks of Adobe Systems Incorporated in the United States and/or othercountries.
Oracle is a registered trademark of Oracle Corporation.
UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.
Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin aretrademarks or registered trademarks of Citrix Systems, Inc.
HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, WorldWide Web Consortium, Massachusetts Institute of Technology.
Java is a registered trademark of Sun Microsystems, Inc.
JavaScript is a registered trademark of Sun Microsystems, Inc., used under license fortechnology invented and implemented by Netscape.
SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer,StreamWork, and other SAP products and services mentioned herein as well as theirrespective logos are trademarks or registered trademarks of SAP AG in Germany and othercountries.
© 2014 SAP AG. All rights reserved
Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, CrystalDecisions, Web Intelligence, Xcelsius, and other Business Objects products and servicesmentioned herein as well as their respective logos are trademarks or registered trademarksof Business Objects Software Ltd. Business Objects is an SAP company.
Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybaseproducts and services mentioned herein as well as their respective logos are trademarks orregistered trademarks of Sybase, Inc. Sybase is an SAP company.
All other product and service names mentioned are the trademarks of their respectivecompanies. Data contained in this document serves informational purposes only. Nationalproduct specifications may vary.
The information in this document is proprietary to SAP. No part of this document may bereproduced, copied, or transmitted in any form or for any purpose without the express prior
written permission of SAP AG.
This document is a preliminary version and not subject to your license agreement or anyother agreement with SAP. This document contains only intended strategies, developments,and functionalities of the SAP® product and is not intended to be binding upon SAP to anyparticular course of business, product strategy, and/or development. Please note that thisdocument is subject to change and may be changed by SAP at any time without notice.
SAP assumes no responsibility for errors or omissions in this document. SAP does notwarrant the accuracy or completeness of the information, text, graphics, links, or other itemscontained within this material. This document is provided without a warranty of any kind,either express or implied, including but not limited to the implied warranties ofmerchantability, fitness for a particular purpose, or non-infringement.
SAP shall have no liability for damages of any kind including without limitation direct,
special, indirect, or consequential damages that may result from the use of these materials.This limitation shall not apply in cases of intent or gross negligence.
The statutory liability for personal injury and defective products is not affected. SAP has nocontrol over the information that you may access through the use of hot links contained inthese materials and does not endorse your use of third-party Web pages nor provide anywarranty whatsoever relating to third-party Web pages.