ema presentation: driving business value with continuous operational intelligence
TRANSCRIPT
1
Driving Business Value With Continuous Operational Intelligence
Today’s Presenters Jim Frey Vice President of Research, Network Management Jim has over 25 years of experience in the computing industry developing, deploying, managing, and marketing software and hardware products, with the last 20 of those years spent in network and infrastructure management, straddling both enterprise and service provider sectors.
Erik Giesa SVP of Marketing and Business Development, ExtraHop Networks Erik guides market strategy and execution with a focus on helping customers transform their IT operations. Erik offers keen insight into the goals and requirements of enterprise IT organizations and ensures ExtraHop meets those needs. Erik has held executive positions in product management, marketing, solutions architecture, and business development for companies like F5 Networks, Holistix, WRQ, and hDC Express.
Agenda
• Moving from Ops Monitoring to Continuous Ops Intell • ExtraHop Networks: Wire Data Analytics Solutions • Three Key Steps to Continuous Operations Intelligence
• Finding the Right Data Set • Turning Data into Operational Intelligence • Sharing the Results
• Wrap-up and Key Takeaways • Q&A
Slide 3 © 2014 Enterprise Management Associates, Inc.
Moving from Operations Monitoring to
Continuous Operational Intelligence
© 2014 Enterprise Management Associates, Inc.
The Big Picture
IT has become essential business-enabling mechanism • Datacenter/network provides hosting & delivery for applications • All orgs use apps & IT services as basis of work/business
processes
And so….
• IT Ops must establish visibility into health and operations of essential application/business infrastructure.
The challenge
• Finding relevant insights • Keeping up w/ speed of actual business activity
Slide 5 © 2014 Enterprise Management Associates, Inc.
IT Ops Moving Towards Service, Application Orientation – Even the Network Team!
Slide 6 © 2014 Enterprise Management Associates, Inc.
Which are becoming more important to the network management team?
66%
59%
55%
55%
37%
4%
0% 10% 20% 30% 40% 50% 60% 70%
Service Quality
End User Experience
Application Performance
Problem Prevention
Internal SLAs
None of the above
Source: Managing Networks in the Age of Cloud, SDN, and Big Data: Network Management Megatrends 2014, Enterprise Management Associates, April 2014 Sample Size = 246
Why Operational Intelligence?
IT Operations Monitoring is already valuable • Especially App-aware/Transaction-centric monitoring • Helps connect IT to business priorities
But there’s much more value to be gained • Insights directly into business activity • Transaction types, rates and results
Opportunity • Transform IT Monitoring into Business Aware Monitoring
Slide 7 © 2014 Enterprise Management Associates, Inc.
Traditional Options for Business-Aware Monitoring
Business Service Mgmnt (BSM) • “Top-Down” approach
• Gather data via multiple underlying “domain” systems
• Build models and write rules to correlate/normalize
• Advantages: • Can be very complete, rigorous
• Disadvantages: • Very expensive to deploy,
maintain • Near real-time, at best
Business Intelligence (BI) • “Data-centric” approach
• Dump all available data into very large database (Big Data)
• Run periodic or ad hoc queries
• Advantages:
• Can reveal important/surprising insights
• Disadvantages:
• Not real-time
© 2014 Enterprise Management Associates, Inc.
What is Operational Intelligence?
• Start with IT Operational Monitoring Data
• Find Business relevant indicators and metrics
• Transform into information by data enhancement
• Apply analytics to elicit actionable results
• Share the findings, and…..
• Do it all at the speed of business: • REAL-TIME • CONTINUOUS
Slide 9 © 2014 Enterprise Management Associates, Inc.
ExtraHop Networks Introduction
“With ExtraHop, we’ve achieved the ‘holy grail’ of IT operations. We’re not just
remediating problems faster, but preventing problems from occurring in the first place.”
— VP of Technical Operations
Blue-Chip Customers
Technology Partners
Industry Recognition
• Disruptive platform that enables greater visibility, insight and IT operations intelligence
• Technology leadership in analyzing wire data
• Monitoring over 1M systems and trillions of transactions daily
• Founded in 2007; rapidly emerging leader
Java/.NET, enterprise apps, custom apps, middleware
Oracle, SQL Server, DB2, Informix, MySQL, Postgres, Sybase
SAN, NAS
Authentication, DNS, FTP
Firewalls, load balancers, WAN accelerators, switches, routers
Fat clients, web browsers, mobile devices, VDI clients
SOAP/XML, JSON, AWS (EC2/SQS/S3), CICS, X12, AS2, Riak
Apache, IIS
Everything Communicates on the Wire
What is baseline performance? What is the impact of this code update in production?
Which queries are running slow? Which methods are used? How does this schema change affect performance?
What are file access times? Which users are accessing sensitive files? Are my files exposed?
Is authentication set up correctly on all systems? Is there a DNS misconfiguration?
How well are applications using the network? How well is the network delivering applications?
Which users and client types are affected? What are users doing on the network?
Which web services are broken? Which applications are affected? Am I detecting anomalous behaviors?
Which servers are slow? What are the error messages? Web Tier
App Tier
Database Tier
Storage Tier
Shared Services
Network Tier
Clients
Web Services
External APIs
Three Key Steps to Continuous Operational Intelligence
Many Choices for Operations Monitoring Data, but Not All Fit the Bill for Operational Intelligence
Step #1: Finding the Right Data Source
• SNMP/WMI Polling: Not granular enough, not real-time • Logs: Very granular, but incomplete/inconsistent • Synthetic test: Helpful for prevention/early warning, but not real
business activity • Wire data: Rich, real-time, all activity
Slide 14 © 2014 Enterprise Management Associates, Inc.
Wire Data: Great Source, if You Can Handle it!
Challenges with Wire Data • High speed/volume: Often 10Gbps+
• Low Signal/Noise: Must find relevant details
• Hidden: Can require decryption
Slide 15 © 2014 Enterprise Management Associates, Inc.
ExtraHop’s Wire Data Analytics Platform
L2–L7 Packet Data Structured Wire Data
ExtraHop’s Real-Time UI Real-Time Stream Processor
Real-Time Stream-Processing for Data-Driven Operations Delivers Tremendous Value:
• Cross-Tier Transaction & User Performance • Rapid Problem Identification & Resolution • Real-Time Business Analytics • Anomaly Detection & Security Monitoring
Core ExtraHop Value: Data Transformation From Raw
Unstructured Data
To Structured Wire Data
The Source of Truth and Insight
Step #2: Turning Monitoring Data into Operational Intelligence First: Enhance wire data via additional info sources General
• Translate IP addresses into human/system names • Translate protocols into application names • Add geo information • Etc…..!
Application/Business-specific • Looking up transaction types from codes • Finding customer names from codes • Cross-referencing product types/families from SKUs • Etc…..!
Slide 18 © 2014 Enterprise Management Associates, Inc.
Step #2: Turning Monitoring Data into Operational Intelligence, continued…
Next: Apply Analytics • Track long-term behaviors and trends
• Statistical Modeling • Dynamic Thresholding
• Identify anomalous levels, events • Automation is essential!
Slide 19 © 2014 Enterprise Management Associates, Inc.
Role-Based & Time-Based Data Visualization
Role-based visualization
Time-based comparison. What happened yesterday compared to now?
Frequency-based comparison. What are the most frequently accessed files?
What are the best and worst performing systems? Are they within my SLAs?
Rapid Analysis & Visualization
Simply explore all metrics you want to visualize, compare, overlay, or trend.
Understand the relationship of the top-level metric category with details. Custom
metrics are treated as first order metrics!
Rapid Analysis & Visualization
1. Search and add metric source
2. Select associated detail metric(s)
3. Add to dashboard
Rapid Analysis & Visualization
Enriched Insight: Open Data Context API
The Open Data Context API enables customers to inject information from a wide range of third-party sources (e.g. user IDs) into ExtraHop’s session table, giving wire data more context. The API is bi-directional and also allows external sources to pull information from ExtraHop’s session table.
Turning Monitoring Data into Operational Intelligence
IT Director – Payment Processing Co.
Is there a correlation between my order transaction performance by merchant and revenue? Can I capture real-time
order information without changing my apps or creating a rigid, slow, and
expensive BI architecture?
ExtraHop Wire Data Stream Processing
From this: REAL-TIME WIRE DATA STREAM PROCESSING of any raw bytes off the wire into structured data that can be measured, visualized, alerted upon, and trended. ExtraHop is the only modular and programmable Wire Data analytics platform in the industry.
Customer Requirements:
• Surgically collect and measure only these elements, no more Big Data garbage.
• Do it with zero changes to my servers, apps, DBs, or infrastructure.
• Implementation should take minutes, not months or years.
• I want the option to stream this data and any other to a non-proprietary NoSQL data store to combine w/ other data sets.
ExtraHop: Wire Stream Processing in Action
IT Director – Payment Processing Co.
In less than 30 minutes, I wrote an Application Inspection Trigger and
ExtraHop is correlating order transaction performance with all unique transactions,
orders, and revenue by merchant.
Step #3: Sharing the Results
What Can You Do With Operations Intelligence?? • Adapt for Key Constituencies
• Personnel: Line of Business / Division / Department • Systems: Big Data, Business Intelligence
• What is needed • Flexible, intuitive live dashboards • Easily consumable reports • APIs, data gateways for direct sharing
Slide 28 © 2014 Enterprise Management Associates, Inc.
Real-Time Health Care Analytics
ExtraHop’s out-of-band, real-time parsing of HL7 messages enables a
faster, more accurate, non-invasive, and extremely cost effective mission operations
analysis platform. Can easily be done by location and any attribute
found in the HL7 message.
Enhanced Collaboration Personal and Shared Dashboards
Copy/Create/Share Edit/Move Filter
Non-Proprietary NoSQL DB
Open Sharing with Other Systems
REAL-TIME stream processing, analysis, and visualization
POST-HOC Multi-dimensional analysis
AND/OR
Precision Transaction Streaming
Chartio
Visualization Tools
• Application teams • DBAs • Network team • Security team • Virtualization
team • Business owners • … and more
We Believe Data Should Be Set Free
Wire Data
Machine Data
Agent Data
Synthetic Data
Human-generated Data
Open Source NoSQL DB
Open ITOA Principles & Benefits
• Non-proprietary db: No vendor lock-in • High-performance and scale • Non-invasive precision data collection
• Lower costs: No data charge for use or growth • Flexible data exploration / analysis • Rapid and simple deployment
The Need for Storing & Querying Wire Data Transactions
IT Director – Payment Processing Co.
Finance called and said one of our merchant customers is complaining that we’re creating duplicate orders. Their
customers are upset about over-charging. They’ve threatened to move to another
clearinghouse.
Is our payment processing application and engine broken?
How exposed are we to SLAs?
Which of their customers did this happen to, when and
how many?
How much revenue was involved?
Is it just this one merchant or are there
others?
Streaming Intelligence to Open Data Stores Limited only by the NoSQL DB’s sharding (clustering capability), ExtraHop can stream an unlimited number of cross-tier transactions (up to 400,000 per second from one appliance). All transactions are time-stamped and can be stored for any transaction, protocol and / or payload type for post-hoc and multi-dimensional analysis.
All transactions are pre-processed and surgically extracted eliminating Big Data garbage. Streamed Wire Data is stored at no additional cost from ExtraHop. Use , combine, and grow data without fear.
A Simple Query Answering Hard Questions
Director of IT – Payment Processing Co.
A duplicate order search of ExtraHop’s Wire Data in my NoSQL DB is a simple query
across tens of millions of records and I don’t have to pay for the growth and use of this
data.
Rapidly Answering Near-Impossible Questions
Is our payment processing application and engine broken? It’s not the payment processing app. Duplicate orders are only being processed from a single merchant indicating the problem is on their end. Also, in ExtraHop’s real-time dashboards, it shows all transactions have been processed without errors.
Is it just one merchant or are there others we don’t yet know about? It’s only one merchant #9145290 which is Acme Inc., the one who called Finance.
Rapidly Answering Near-Impossible Questions
Is it just one merchant or are there others we don’t yet know about? It’s only one merchant #9145290 which is Acme Inc., the one who called Finance.
Which of their customers did this happen to, when, and how many? Only 2 customers were affected. Interestingly, both customers purchased an item on the exact same data and time, 08/04/2014 at 5:51 PM. A call to the merchant revealed that was when IT was cutting over two e-commerce apps inherited from their recent acquisition of Zexel Corp. In the process, one of the appsmust have allowed multiple commits from an impatient user pushing the submit button more than once.
How exposed are we to SLA penalties and how much revenue was involved? There will be no SLA penalties especially since we isolated the problem in under 10 minutes. Overcharges totaling $15K were involved.
EMA Key Takeaways
• IT Operations Monitoring can evolve/transcend traditional functions to address direct, real-time business monitoring
• Find a data set, such as wire data, that can provide the insights you need
• Translate operational monitoring metrics into business relevance via data augmentation and analytics
• Share the results for best leverage
Slide 38 © 2014 Enterprise Management Associates, Inc.
Q&A Try the online interactive demo at http://www.extrahop.com/enterprise/start/