what an it leader needs to know about bigdata (it leaders event september 2015)
TRANSCRIPT
03/05/2023 1
WHAT THE IT LEADER NEEDS TO KNOW ABOUT BIG DATA
John MortonExecutive Leaders NetworkSeptember 2015
Executive Leaders Network – September 2015
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 2
In 2011 (Gartner) this now includes – Complexity!! May 2011 McKinseys Report “Big Data : The next frontier for Innovation,
Competition and Productivity” stated : US healthcare reduction by $300 B a year 2/3rds from a 8% reduction
in national healthcare Retailers can increase operating margin by 60% by fully utilising data Euro 100 Bn reduction in Government administration across Europe
Big data is a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target currently ranging from a few dozen terabytes to many petabytes of data in a single data set. (Wikipedia) In a 2001 (Gartner), defined data growth challenges (and opportunities) as being three-dimensional, increasing volume (amount of data), velocity (speed of data in/out), and variety (range of data types, sources).
WHAT IS BIG DATA?
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 3
Predict ActAct
Appliances
Structured
SAP
Internet
Windows
Semi-Structured
Other Sources
Un-Structured
Real Time
High Volume Batch
Web Services
Message Queues
Sense
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 4
USING ANALYTICS
Com
petit
ive
Adva
ntag
e
Intelligent insight
Raw Data
Business Maturity
Cleaned Data
Standard Reports
Cubes and dashboards
Alerts
Explore
Predict
Optimise
What has happened?
What is the frequency?
Where is the problem?
What actions are needed?
What is the best outcome?
What may happen next?
Why is it happening?
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 5
WHAT DOES DO?
Analytics
Data Intelligence
Customer Intelligence
Business Intelligence
Fraud and Risk
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 6
EIM FRAMEWORKVision
& Strate
gy
Governance
EIM Core Processes Organization
Tools & Technologi
es
Vision
Mission
StrategyGoals
& Objectives
Sponsorship
Stewardship
Policies, Principle
s &Tenets
Alignment
Structure
Data QualityData IntegrityData Security & Protection
Data LifecycleData
MovementSemantics Management
Database ManagementMaster Data ManagementInformation
Services (A&R)Services &
Support
Value Proposi
tions
CSFs & KPIs
Structure (Virtual,
Hybrid, …)Roles &
Responsibilities
Functional ServicesBusiness Value &
Relationship Managemen
t
Information
Architecture
Blueprint Manageme
nt
Technologies (DBMS, CMS, ETL, EAI, EII,
Data Modeling,
BI/DW, Collaborati
on)Knowledge
base & Repositori
esStandards
& Best Practices
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 7
DATA LAKE
IT Business
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 8
DATA LAKE FRAMEWORKVision
& Strate
gy
Governance
EIM Core Processes Organization
Tools & Technologi
es
Vision
Mission
StrategyGoals
& Objectives
Sponsorship
Stewardship
Policies, Principle
s &Tenets
Alignment
Structure
Data QualityData IntegrityData Security & Protection
Data LifecycleData
MovementSemantics Management
Database ManagementMaster Data ManagementInformation
Services (A&R)Services &
Support
Value Proposi
tions
CSFs & KPIs
Structure (Virtual,
Hybrid, …)Roles &
Responsibilities
Functional ServicesBusiness Value &
Relationship Managemen
t
Information
Architecture
Blueprint Manageme
nt
Technologies (DBMS, CMS, ETL, EAI, EII,
Data Modeling,
BI/DW, Collaborati
on)Knowledge
base & Repositori
esStandards
& Best Practices
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 9
DATA LAKE FRAMEWORK CAPABILITIES
INFORMATION GOVERNANCESupporting
ServicesEvents, linkage
Workflow & Business
Rules
Information Quality
and Information metrics
MDM
INFORMATION AND CONTENT ACCESS
INFRASTRUCTURE SERVICESSecurity, Meta-data & Lineage, Versioning, Monitoring &
Deployment
Algorithm Managemen
t, Calibration
& Monitoring
Information and
Algorithm Assessment
INSIGHT SERVICES
Interfaces and
Integration
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 10
VALUE OF DATA LAKESC
umul
ativ
ebu
sine
ss v
alue
time
uncertainty decreases over time
unce
rtain
ty
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 11
TWO WORLDSInformation
Prep
Information Analyse
Develop Model
Calibrate Model
Information PreparationInformation ExplorationInformation VisualizationReport Creation
Exploratory AnalysisWhat if assessmentCorrelationsDescriptive SegmentationPredictive Modeling
DataAnalyst
AnalyticsAnalyst
Industrialise Model
Validate Model
Deploy Model
Evaluate and Calibrate
Model ValidationModel DeploymentModel Monitoring Information Preparation
IT Operations
Evaluates processesAssesses EffectivenessCalibrates model
Business Operations
Frames Problem in contextEstablishes metrics of successAgrees outcomes
Business Analyst
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 12
CEO
Function
Function
Function
Function
CEO
Function
Function
Function
Function
CEO
Function
Function
Function
Function
Function
Leading analytics CMO + CIO + CPO + COO £
ORGANISATION STRUCTURES
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 13
BUSINESS ANALYTICS ECO-SYSTEM
Data assets
Analyticsstrategy
Value propositions
ICT strategy
HR strategy
Business strategy
Vidgen, R., (2014). Creating business value from Big Data and business analytics: organisational, managerial and human resource implications. Hull University Business School Research Memorandum, no. 94, ISBN 978-1-906422-31-8.
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 14
WHAT ARE YOU MISSING?
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 15
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 16
PARTING THOUGHTS1. Can I trust that the technology chosen is
going to get the best out of my data?2. What are the trade-offs between the
algorithms employed by “service A” versus “service B”?
3. If the solution claims to be automated analytics, how robust is that automation?
4. Is the technology “extensible” and “interoperable” to meet changing demands from data, evolving business, or competition?
Copyright © 03/05/2023, Computers, Processes and Management Limited.. All rights reserved.
Executive Leaders Network – September 201503/05/2023 17
THOUGHTS ?
Eur. Ing. John Morton BSc, CEng. FBCS, CITP, [email protected] +44 7771 740203Thanks to Pixabay for Images