data est, inepte! copyright 2005 afcea portugal afcea technet lisbon 2005
TRANSCRIPT
Data Est, Inepte! Copyright 2005
AFCEA Portugal
AFCEA TechNet
Lisbon 2005
Data Est, Inepte! Copyright 2005
AFCEA Portugal
“Data Inepte”
It’s the data, dummy!
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Role of the Computer
Is it:
a) To assist human beings to take decisions?
b) To provide an expensive desk top accessory?
c) To provide employment for the IT department?
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Answer :
Almost invariably b) and C)
Why?
Because we are creatures of habit and dislike change!
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Historical Perspective
• In the beginning was paper!
• Then - computer automation– BUT memory very expensive– Hence RDBMS and tricks of saving space
• Computer storage now very cheap
• BUT – we use same tricks.
• WHY?
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Obsessions
• People are obsessed with Relational database models
• Major companies make their money out of RDBMS.• We have millions of information silos throughout the
government and commercial world• Over 30 years, no common data standards have been
achieved.• Mergers and acquisitions create complex information
environments• Very public failures: IR, CSA, Police, MoD, etc.
Data Est, Inepte! Copyright 2005
AFCEA Portugal
•24% decisions are wrong
•Average annual cost/company is £800,000
•Requirement to be agile
•Collaboration preferred
•Reluctance to take risks, determination to seek optimum solution or stick by a decision!
Business Decisiveness Survey: (Capgemini: August 2004)
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Cost of Computing
Cost of Memory
Cost of Hardware
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Cost of Computing
Cost of Memory
Cost of Hardware
Systems Costs
Data Est, Inepte! Copyright 2005
AFCEA Portugal
High Costs
• The problem is Legacy systems and the RDBMS:– Cost of scrubbing and cleaning data– Normalisation and renormalisation– New business process-led initiatives vs. IT
driven projects
“Business data is more often searched and analysed than inserted.” - IBM
Data Est, Inepte! Copyright 2005
AFCEA Portugal
What can we do?
• Modern IT systems focus on the “system” rather than the “data”.
• Must shift the focus from the IT department to the Boardroom.
• Operational imperatives demand that we focus on the data – not the system.
“Data inepte”
It’s the data, dummy!
Data Est, Inepte! Copyright 2005
AFCEA Portugal
What about legacy systems?
• Legacy systems range from those that are old and unused any more to those which are still in active use.
• Traditional answer was to “migrate” systems to new standards.
• Legacy systems continue to be of utility.• Need to build on them – not destroy them!• Must have lower costs, faster delivery and
improved confidence.
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Importance of Data
DATA
Intelligence
Governance
Compliance
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Governance, Compliance and Intelligence
• Essential in any organisation, including in military operations:
• Governance: “How an organisation delivers its strategic goals within the environment within which it operates.”
• Compliance: “How an organisation manages its information environment in such a manner as to meet all regulatory demands placed upon it.”
• Intelligence: “How an organisation develops a clear strategic or tactical picture of its operational environment.”
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Basing Decisions on the Intelligent Use of Data
• Importance of “context” as well as “content”.• Development of clear intelligence “threads”• Depth of intelligence as well as breadth• Need to get an “integrated” picture from
disparate legacy systems• In the police case: address – cars, guns, who
lives next door, bank accounts, previous form, etc
Data Est, Inepte! Copyright 2005
AFCEA Portugal
What’s the answer?
• Use of XML native database approach.• This approach has been endorsed by IBM with
Viper project – a native XML extension to DB2.• Fast, reliable, multi data-source with improved
confidence of assurance of outcome.• Particularly relevant to intelligence processes,
especially where multitude of sources.• Police proof of concept underway.
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Legacy Systems
Legacy Systems: structured and unstructured data
TyrrellSmith XML ‘Rummage’ Store
Applications (of any sort)
Data Est, Inepte! Copyright 2005
AFCEA Portugal
“Actionable Data”
• Data which gives that intelligence including all key information for a decision to be made.
• Clear and uniquivocable?• Adequate provenance (if necessary to satisfy
the CPS or equivalent)• Inevitably from an increasingly large database
pool. • Requirement to be cross-organisational.• Must be fully auditable.
Data Est, Inepte! Copyright 2005
AFCEA Portugal
The Intelligence of Intelligence
• Must be a clear understanding of the nature of information – its strengths and limitations.
• Recognise database limitations
• The “knowns” and the “unknowns”
• Clear analytical skills.
• Ability to conduct “what if?” scenarios.
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Seeing is believing…… or is it?
• Panorama 1963
• Chinese Embassy, Belgrade
• WMD – Iraq 2003
• Understand perceptions.
• We often see what we want to see.
• Interaction of humans and technologies.
Data Est, Inepte! Copyright 2005
AFCEA Portugal
The Humble Carbon Unit
Data Est, Inepte! Copyright 2005
AFCEA Portugal
The Humble Carbon Unit
Lateral prefrontal cortex
Trade offsSophisticated decision makingAbstract rewards
Limbic system
PrimitiveImmediate stimuliImmediate decisions
Data Est, Inepte! Copyright 2005
AFCEA Portugal
The Humble Carbon Unit
Lateral prefrontal cortex
Trade offsSophisticated decision makingAbstract rewards
Limbic system
PrimitiveImmediate stimuliImmediate decisions
Data Est, Inepte! Copyright 2005
AFCEA Portugal
The Humble Carbon Unit
Lateral prefrontal cortex
Trade offsSophisticated decision makingAbstract rewards
Limbic system
PrimitiveImmediate stimuliImmediate decisions
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Agile Data
• Understand the timelines within which data is – Valid– Relevant
• Focus on key issues BUT remember that too much focus can miss the essential facts.
• Require a sophisticated review process to ensure appropriate data available.
• Use of the traditional OODA loop.• Validity of data, irrespective of data source.
Data Est, Inepte! Copyright 2005
AFCEA Portugal
The OODA Loop
Data Est, Inepte! Copyright 2005
AFCEA Portugal
Summary
• Move from a “system centric” to a “data centric” approach.
• Linkage of data silos using native XML processes.• Develop agility of data management.• Recognise strengths (and limitations!) of the human
element.• Improve governance, compliance and intelligence
capabilities of organisations.• Give the “war fighter” that joined up data he or she
requires, where it is required at the time it is needed!