what big data means to you? - brianmurrow.combrianmurrow.com/portals/0/publications/brian murrow -...
TRANSCRIPT
© 2013 IBM Corporation
What Big Data Means to You?
March 21, 2013
Brian D. Murrow
Associate Partner, Business Analytics and Optimization
2 © 2013 IBM Corporation
Table of contents
� Challenges & opportunities facing federal
organizations
� Conceptual Framework
– Big data
– Advanced analytics
– Reporting and operational insights
� Component Framework
� 6 ways we can make a difference today
What Big Data Means to You,
March 21, 2013.
Objective: Big Data -- Let’s do a deep dive
3 © 2013 IBM Corporation
Challenges and opportunities facing federal organizations
What Big Data Means to You,
March 21, 2013.
4 © 2013 IBM Corporation
We’ve seen changes like this before ……
Time
Pla
tfo
rms &
Bu
sin
ess V
alu
e
Transaction Systems
Web, e-business and SOA
New Era Analytics,
brought to us via Big Data
1960-1980s 1990-2000s 2010s
What Big Data Means to You,
March 21, 2013.
5 © 2013 IBM Corporation
What is Big Data and Advanced Analytics?
Structured Data & Unstructured Content
Descriptive Analytics
Prescriptive Analytics
Predictive Analytics
Made consumable and accessible to everyone, to provide business insights through:
What if these trends
continue? Forecasting
How can we achieve the best
outcome and address
variability? Stochastic
Optimisation
What is happening
What exactly is
the problem?
How many, how often,
where?
What actions are needed?
What could happen?
Simulation
How can we achieve the best
outcome? Optimisation
What will happen next if?
Predictive Modelling
Extracting insight,
concepts and
relation-ships
Content Analytics
Deep insights
On Sentiment
WebAnalytics
Big Data → Advanced Analytics → Business Insights
What Big Data Means to You,
March 21, 2013.
6 © 2013 IBM Corporation
The Characteristics of Big Data include:
Collectively analyzing the broadening Variety
Responding to the increasing Velocity
Cost efficiently processing the growing Volume
Establishing the Veracity of big data sources
1 in 3 business leaders don’t trust the information they use to make decisions
50x 35 ZB
20202010
30 Billion RFID sensors and counting
80% of the
worlds data is unstructured
What Big Data Means to You,
March 21, 2013.
7 © 2013 IBM Corporation
Operational considerations as it related to supporting big data.
7. Up-time reliability
1. Hosting facility physical location
2. Massively Parallel Processing (MPP)
3. Physical Security
4. Data-level security
6. Platform scalability and
extensibility
10. Cost versus value
8. Data-level and application
support
9. Service level agreement and accountability
5. Data Governance
What Big Data Means to You,
March 21, 2013.
8 © 2013 IBM Corporation
Advanced analytics solutions requires several interacting designsteps. Notional Information Flow – The Information Supply Chain.
What Big Data Means to You,
March 21, 2013.
→ Operationalizing Analytics →
9 © 2013 IBM Corporation
The use of advanced analytics provides a great degree of business value when combined with big data capabilities.
Degree of Complexity
Com
petitive A
dvanta
ge &
Busin
ess V
alu
e
Standard Reporting
Ad hoc reporting
Query/drill down
Alerts
Simulation
Forecasting
Predictive modeling
Optimization
What exactly is the problem?
What will happen next if ?
What if these trends continue?
What could happen…. ?
What actions are needed?
How many, how often, where?
What happened?
Stochastic Optimization
Descriptive
Prescriptive
Predictive
How can we achieve the best outcome?
How can we achieve the best outcome including the effects of variability?
What Big Data Means to You,
March 21, 2013.
10 © 2013 IBM Corporation
Considerations as it related to analytics:
CRISP-DM Process
What Big Data Means to You,
March 21, 2013.
11 © 2013 IBM Corporation
6 ways government employees can make a difference today.
1. Improper Payments
2. Regulatory Oversight
3. Domestic Security
4. Operational Efficiency
5. Prescriptive Policy Analysis
6. Anyone Can Make A Difference
What Big Data Means to You,
March 21, 2013.
12 © 2013 IBM Corporation
The IBM Big Data Platform
BI / Reporting
BI / Reporting
Exploration / Visualization
FunctionalApp
IndustryApp
Predictive Analytics
Content Analytics
Analytic Applications
IBM Big Data Platform
Systems Management
Application Development
Visualization & Discovery
Accelerators
Information Integration & Governance
HadoopSystem
Stream Computing
Data Warehouse
What Big Data Means to You,
March 21, 2013.
13 © 2013 IBM Corporation
The IBM Big Data Platform
BI / Reporting
BI / Reporting
Exploration / Visualization
FunctionalApp
IndustryApp
Predictive Analytics
Content Analytics
Analytic Applications
IBM Big Data Platform
Systems Management
Application Development
Visualization & Discovery
Accelerators
Information Integration & Governance
HadoopSystem
Stream Computing
Data Warehouse
• SPSS• IBM Content
Analytics• System T
• InfoSphere Data Explorer
• i2
• InfoSphere BigInsights
• InfoSphere Information Server
• InfoSphere Data Explorer
• Cognos• i2• InfoSphere Data
Explorer
• InfoSphere Streams
• PureData for:• Transactions• Analytics• Operational
Analytics• InfoSphere
Warehouse
What Big Data Means to You,
March 21, 2013.
14 © 2013 IBM Corporation
MIDAS Architecture: A detailed view from IBM research
14
DB Export
Index
Generation
BigInsights
GPFS
DE SystemT
Extraction & Integration Flow
Analytic
s
DB
Analytic +
Search UI
Analytics
Reports
Applications
MIDAS
Platform
CrawlCrawl
Global
Resolve
Global
Resolve
Global
Resolve
Global
Resolve
Map/FuseMap/Fuse
Map/FuseMap/Fuse Temporal
Analyze
Temporal
Analyze
Temporal
Analyze
Temporal
Analyze
ExtractExtract Local
Resolve
Local
Resolve
Local
Resolve
Local
Resolve
Index
Global
Resolve
Global
Resolve Map/FuseMap/Fuse Temporal
Analyze
Temporal
AnalyzeLocal
Resolve
Local
Resolve1
2
n
MIDAS offers analytical insights into public financial filings by exposing information concepts
and relationships within extracted concepts, enabling:
•Extraction, mapping, temporal analysis, and fusion, creating person and organization entities
•Analysis of person-level data and person-to-organization relationships
•Analysis of organization counter-party relationships
•Advanced quantitative analytics on fused structured and structured data
What Big Data Means to You,
March 21, 2013.
15 © 2013 IBM Corporation
MIDAS: Explore counterparty relationships with other companies
1515
e.g. observe shared director of Citigroup with Federal National Mortgage Association, Diana Taylor
e.g. employment overview of insiders of Citigroup
Relationships aggregated from multiple forms (10-K, SC 13G, SC 13D, 13F, 3, 4, 5, FDIC Call Reports)
What Big Data Means to You,
March 21, 2013.
16 © 2013 IBM Corporation
1616
MIDAS Analytics: Analyze and aggregate recent loans issued by counterparty
Loan details : lender names, commitments
Link to borrower’s original SEC filing
Joint loans issued with other lenders
Recent loans issued by Citigroup’s subsidiaries
e.g. Loans issued by Citigroup
Key financial metrics for banking subsidiaries
Breakdown of overall loans outstanding by subsidiary
Breakdown by type of loan
e.g. drill down on Citigroup loan activity by subsidiaries (from FDIC Call Records)
What Big Data Means to You,
March 21, 2013.
17 © 2013 IBM Corporation
What Big Data Means to You,
March 21, 2013.