kai wähner – real world use cases for realtime in-memory computing - nosql matters barcelona 2014
TRANSCRIPT
In-Memory Computing “Real World Use Cases”
Kai WähnerTechnical [email protected]@KaiWaehnerwww.kai-waehner.deLinkedIn / Xing à Please connect!
Kai Wähner
Consulting Developing Coaching Speaking
Writing Selling
Main Tasks Requirements Engineering
Enterprise Architecture Management Business Process Management
Architecture and Development of Applications Service-oriented Architecture
Integration of Legacy Applications Cloud Computing
Big Data
Contact Email: [email protected] Blog: www.kai-waehner.de/blog
Twitter: @KaiWaehner Social Networks: LinkedIn, Xing
Disclaimer
!
These opinions are my own and do not necessarily represent my employer
Key Messages
In-Memory Computing is used for Acting in Real-Time!
In-Memory Computing is NOT just Caching!
Eventing and Fault-Tolerance move In-Memory to another Level!
© Copyright 2000-2014 TIBCO Software Inc. 5
Agenda
• Introduction to In-Memory Computing• Use Cases / Customer Success Stories
© Copyright 2000-2014 TIBCO Software Inc. 6
Agenda
• Introduction to In-Memory Computing• Use Cases / Customer Success Stories
Time
Business Value
Business Event
Data Ready for Analysis
Analysis Completed
Decision Made
$$$$
$$$
$$
$ Action Taken
Business Value of Events over Time
In-Memory Computing and Event Processing
speeds action and increases business
value by seizing opportunities while
they matter
© Copyright 2000-2014 TIBCO Software Inc. 8
• Hardware costs declining• Data Processing Requirements
exploding• Traditional Approaches not
scaling– Relational Databases– Clustered Databases– In-Memory Caches– Messaging Systems
Drivers for In-Memory Computing
© Copyright 2000-2014 TIBCO Software Inc. 9
Database Landscape in 2014
h9p://blogs.the451group.com/ informaCon_management/2014/03/18/ updated-‐data-‐plaIorms-‐landscape-‐
map-‐february-‐2014/
© Copyright 2000-2014 TIBCO Software Inc. 10
Agenda
• Introduction to In-Memory Computing• Use Cases / Customer Success Stories
LOADER
Caching for Fast Data Access
• Cache to slower systems • Read-‐only • Not the system of record • No persistence required • Side benefit: Backend load
is reduced
SERVICE
Caching + Dynamic Load
• Dynamically loaded into Memory when the data is first accessed by a client applicaCon
• Service can present a standard interface
• Client applicaCons are not required to implement any In-‐Memory specific code
(1) Check Cache
(2) Load from DB if not in Cache
Routing Messages to Back-Office Applications
• Receive a common data feed that needs to be parsed and routed to several back-‐office applicaCons can use
• In-‐Memory holding reference informaCon for the rouCng applicaCon. The router can quickly determine where to send the data.
• Examples: Bank payments, insurance claims processing
Off-loading expensive systems
Expensive in terms of response Cme and / or transacCon costs!
Personalized Customer Experience
“With 38 million fans, MGM knows how to put its customers first, it takes more than a smile too. Customers want a personalized, tailored experience, one that knows their name and can anCcipate their needs. With the help of TIBCO technologies that leverage big data and give customers a digital idenCty, MGM can send personalized offers directly to customers, save them a seat, and have their favorite drink on the way. With mulCple customer touch points and channels, MGM can reach customers in more ways, and in more places, than ever before.”
h9ps://www.youtube.com/watch?v=X-‐7S3kCOx9k
Latency Problems: • Several Legacy Systems • Processing via ERP, CRM, Host, etc. In-‐Memory: • Enable Real Time • Only customers that have checked in • System of Record
Handling temporary spikes on a slow ‘system of record’
• An In-‐Memory event listener gets noCfied whenever a data value is changed and sends updates through a message queue for updaCng the master system of record.
• The back office system can also be updated through other channels. • Examples: Christmas Shopping in E-‐Commerce, Ticket Sales, Online Bekng
à In-‐Memory as “system of record” à OpConal: PersisCng data on the local file system (rather than requiring a database for persisCng data
Operational Data Store (Local File System)
• Low-‐latency, high-‐throughput operaConal data – Customer data: e.g. account status and balance,
purchase history: real-‐Cme loyalty (promoCons, cross-‐selling), fraud detecCon, ...
– Market data: e.g. risk assessment, porIolio mgmt, producCon output opCmizaCon, buyer-‐seller matching
– Sensor data: e.g. smart metering / grid, public transport safety – Track and trace: e.g. barcode scans, RFID: logisCcs, airlines
• Why In-‐Memory? – Much faster than tradiConal DB, especially many small transacCons (XTP) – State / data management not addressed by messaging soluCons – EvenCng is a first class feature, changes can be ‘pushed’ in real-‐Cme to interested parCes
(subscribe to changes, conCnuous queries) – Provides for distributed process synchronizaCon – Integrated with CEP engine (e.g. TIBCO BusinessEvents)
Operational Data Store (Local File System)
Situation• Master data management system stores over 800 million customer records across more than 30 enterprise apps. • Stores real-time inventory data to enable ‘Buy online and pick-up at store’ and ‘Smart fulfillment’ features
Problem • Due to lack of correlation between Point of Sale data and inventory, the website contained outdated inventory data.
Products were listed as out of stock when there was actually inventory. • Need to leverage store inventory as well as inventory located fulfillment centers
Solution• In-Memory stores real-time inventory data for the website, the fulfillment application, and other applications that need
access to inventory dataBusiness Impact
• Reduction in customer churn• Intelligent fulfillments leading to greater customer satisfaction• Improved overall efficiency of fulfillment centers and store inventory
Retailer: Inventory Management
Distribution of Rapidly Changing Data
à Examples are monitoring data for a power plant, stock market data, telemetry data for a complex system (example, a satellite), or the status and locaCon of packages for a major logisCcs or shipping company.
Telco: Real-Time Offer Generation and Fulfillment by Different Subcontractors
ReloadGive 100 free SMS to subscriber who tops-up
Total: 12 mio top-up / dayPeak: 300 top-up per sec
Purchase 3G PackageCross-sell Voice/SMS package to subscriber who purchases 3G Mobile Package
Total: 3 mio / dayPeak: 50 events per sec
Voice CallGive discount VOIP package to subscriber who makes a IDD call
Total: 200 mio / dayPeak: 12,000 events per sec
SMS UsageGive discounted SMS package to subscriber who sends SMS more than 10 times a day
Total: 750 mio / dayPeak: 27,000 events per sec
Event Cloud
Purchase BB Package
Reload
Voice Call
IDD Call
OnNet Call
SMS Usage
Event Handling and Processing Touchpoint Integration Billing, Offer
Fulfilled
Fulfill SMS Package
Fulfill 3G Package
Fulfill Voice Package
Fulfill SMS Package
46.7 million subscribers 2,000 SMS notifications per
seconds
500 offer fulfillments per
second
Offer MessageReminderMessage
FulfillmentMessage
State-‐full Data
Storing State-full Data for Enterprise Applications
Super Fast Compute Grid for Intermediary Calculations for Analytics
• Technical issues in distributed grid compuCng with large scale data – Work load distribuCon – Process synchronizaCon – Data transfer
• Examples – Risk assessment and management – OpCmizaCon problems: scheduling, cargo assignment, load distribuCon in
power network / grid
• Why In-‐Memory? – Many useful synchronizaCon features (e.g. atomic “take”) – LocaCon transparency and fault-‐tolerance – Real-‐Cme instead of nightly / weekly / ... Data-‐Warehousing approach
Super Fast Compute Grid for Intermediary Calculations for Analytics
Key Messages
In-Memory Computing is used for Acting in Real-Time!
In-Memory Computing is NOT just Caching!
Eventing and Fault-Tolerance move In-Memory to another Level!
Questions? Kai Wähner [email protected] @KaiWaehner www.kai-waehner.de LinkedIn / Xing à Please connect!