gaining real-time data insights and advanced analytics for … · what is trending? 3 internet of...
TRANSCRIPT
Gaining Real-time Data Insights and Advanced Analytics for Financial
Sector
Ghulam Imaduddin
APAC Director of Solution Engineering
October 2017
Banking & Digital Transformation
2
The banking industry in Indonesia is undergoing a significant transformation driven by technology
What is Trending?
3
Internetof Things
The interconnection via the Internet of computing devices embedded in everyday objects, enabling them to send and receive data.
Big Data
It is not just about volume of the data that getting bigger. It’s also about the velocity (speed of incoming data) and variety
Artificial Intelligence
AI is transforming our world. A set of algorithm and science that can predict the future, exposing more opportunity for business
The Benefit
4
SmartDevice
Smart cash machine that can identify the user from the camera. It can make sure the identity to prevent fraud
BetterRiskManagement
Advancing risk analysis with richer information and longer period of data
Accelerating credit scoringprocess from days to hour for better customer experience
Advanced trade decisionby doing NLP and sentiment analysis from scrapped data. Keywords joined with tick data and processed by ML/DL algorithms to calculate risk scores on demand.
Internet of Things
Big Data
Artificial Intelligence
Going More Advance
5
Deep Learning for Mortgage Risk
https://arxiv.org/abs/1607.02470
2005
2008
2010
2012
Deep Neural Network20 Years Mortgage Data, 120 Million Loans
3.5 Billion Records, 2 TB Data300 Explanatory Features
Deep Learning for Trade Decisioning
Deep Neural Net on Limit Order Books1.5 years of equities data50 Terabytes of raw data
Analytics challenges faced by the US Army Intelligence
7
200 sources of streaming data producing 20B new records per day.
Requirements to do ad-hoc analysis with human-response time on hot data.
Reduce reliance on expensive racks of premium hardware.
Kinetica incubated as a massively parallel computational engine for US Army INSCOM
GPU Acceleration Overcomes Processing Bottlenecks
8
3,000+ cores per device in
many cases, versus 16 to 32
cores per typical CPU-based
device.
High performance
computing trend to
using GPU’s to solve
massive processing
challenges
GPU acceleration brings
high performance
compute to commodity
hardware
Parallel processing is
ideal for scanning entire
dataset & brute force
compute.
GPUs are designed around thousands of small, efficient cores that are well suited to
performing repeated similar instructions in parallel. This makes them well-suited to the
compute-intensive workloads required of large data sets.
GPU | Tale of Numbers
9
100x
75%
Performance
100x gains over traditional RDBMS / NoSQL / In-Mem Databases
Cores
Modern GPUs can consist of up to 3000+ cores compared to 32 in a CPU
Costs
75% reduction in infrastructure costs, licensing, staff, etc.
More with Less
Increase performance, throughput, capability while minimizing the costs to support the business
GPUs are designed around thousands of small,
efficient cores that are well suited to performing
repeated similar instructions in parallel. This
makes them well-suited to the compute-intensive
workloads required of large data sets
Performance Increase
Infrastructure Cost Savings
3000vs
32
GPU Database Technology: Features to be Considered
10
Product Maturity andEnterprise Readiness
Easiness of IntegrationWith Another Tools
Data Security, Integrity, and Persistence
In-DatabaseAnalytics
GeospatialSupport
Kinetica Architecture
11
ETL / STREAM PROCESSING
ON DEMAND SCALE OUT +
1TB MEM / 2 GPU CARDS
SQL
NativeAPIs
PA
RA
LLEL ING
EST
Geospatial WMS
Custom Connecto
rs
In-Database Processing
CUSTOM LOGIC BIDMach
ML
Libs
BI DASHBOARDS
BI / GIS / APPS
CUSTOM APPS & GEOSPATIAL
KINETICA ‘REVEAL’
STR
EAM
ING
DA
TAER
P /
CR
M /
TR
AN
SAC
TIO
NA
L D
ATA
UDFs
Risk Management with AI
12
BUSINESS OBJECTIVE
• Move counterparty risk analysis from batch/overnight to a streaming/real-time system for flexible real-time monitoring by traders, auditors, and management
NEW CAPABILITIES DELIVERED
• Ability to handle time-sensitive, compute-intensive risk computations to project years into the future across hundreds of variables
• In-database analytics to run custom XVA algorithms at scale with GPU’s massive parallelization
SOLUTION OVERVIEW
• Kinetica operates as a real-time risk modeling engine running on public cloud-based, Microsoft Azure GPU instances
• Turn-key solution with elasticity, security, ease-of-use, and faster time-to-value
Risk Management
13
STREAM
PROCESSING
ON DEMAND SCALE OUT
IN-DATABASE PROCESSING
MONITORING
Global Positions
Regional Positions
ACCOUNTABILITY
VaR Limits
PRE-TRADE
What-if Scenarios
RISK MANAGEMENT
P&L, Commissions
Sensitivities
Liquidity Risk
Counterparty Risk
Global / Regional
Heads
Desk Managers
Traders
Spot Prices,
Transactions,
Market Risk
Data
External
Transactional
Records
UDF Functions
POSITIONS
ENGINE
CALCULATIONS
PRICING MODELS
RISK
APPLICATIONS
More data available.
Data flowing faster. Many new types of users demanding
sophisticated real-time analysis.
Kinetica
Connectors
ANALYTICS
Trade Decisioning
14
STREAM
PROCESSING
ML L
IBS
ON DEMAND SCALE OUT
IN-DATABASE PROCESSING
APIs
SQL
SQL QUERY
FINANCE
APPLICATIONSFINANCIAL DATA
Tick, Transactional
NEW DATA
Social Media, Forums
Scraped data from blogs,
reports, web, forums…
NLP and sentiment analysis to
classify text and trigger
decisions.
Kinetica
Connector
s
UDF FunctionsKeywords joined with tick
data and processed by
ML/DL algorithms to
calculate risk scores on
demand.
ON DEMAND SCALE OUT +
1TB MEM / 2 GPU CARDS
Fraud Analysis
15
Real-Time Fraud Engines
ML
LIB
S
NativeAPIs
SQL
Fraud
FIN
AN
CIA
L
DA
TA
Cre
dit C
ard
, A
TM
Real-time Financial Data Streams
Key/Value lookups for pattern
analysis. Spatial analytics looking at
time and distance between
swipes/ATM transactions. Ability to
score against trained models.
UDF Functions
BIDMach
Deep Learning APIs for
pattern detection and
predictive analytics. SQL
queries for BI analysis.
ANALYTICS
SQL QUERY
FINANCEAPPLICATIONS
IN-DATABASE PROCESSING
www.kinetica.com
Take the Kinetica Challenge!Try it with your own data
Email: [email protected]
Don’t forget to visit our insightful booth!
DON’T SEND A CPUTO DO A GPU’S JOB