the one question you must never ask!" (information requirements gathering for analytic...
DESCRIPTION
Presentation from 2014 International Data Quality Summit (www.idqsummit.org, Twitter hashtag #IDQS14). Techniques for business analysts and data scientists to facilitate better requirements gathering in data and analytic projects.TRANSCRIPT
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Information requirements gathering for data projects,
Sponsored by W. Edwards Deming
The one question you must never ask!
“If you do not know how
to ask the right question,
you discover nothing.”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Agenda
• Challenges of info requirements gathering
• Methods for engaging with data stakeholders
• Techniques for identifying info requirements
• Establishing actionable business outcomes
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
A bit about me....
• NEW! Research Director, Gartner
• Advisory board member, QFire Software
• 22 years Information Strategy, Data Governance,
Analytics & Business Consulting
– Director of Data Governance at UNSW
– EDS, KPMG, CPW, Acuma, Pelion, SMS
– Scottish Power, United Distillers, O2, Astra Zeneca,
Carphone Warehouse, Vodafone, Riyad Bank
– Commonwealth Bank, NSW Roads & Maritime
Services, Centrelink, OATSIH, NSW Family &
Community Services, CASA, AMSA, FaHCSIA, DAFF,
Navy…
• Information-Management.com “Top 12 on Twitter”
• Best supporting Actor, 2005 Barnet Drama Festival
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Challenges for
information requirements gathering,
Sponsored by Sir Arthur Conan Doyle
“Data! Data! Data! I can’t
make bricks without clay!” (Sherlock Holmes,
The Adventure of the Copper Beeches)
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
“What’s the one
question that you
should never ask?”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
“What information do
you want?!”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Challenges for Information Requirements gathering…
Lack of knowledge about new
information-processing technologies
People don’t know what they want!
Unmet & changing demands
Competition between teams
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
…plus second-guessing future needs.
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Methods for engaging data stakeholders,
Sponsored by Brene Brown
“Maybe stories are just data
with a soul.”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Information Workshop Agenda
• Context setting
• Business Functions
• Challenges
• Opportunities
• Desired Actions
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Don’t just facilitate - provoke!
• “What would happen if…X?”
• “Have you thought about…Y?”
• “Why do you do/want…Z?”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
“Stupid is as stupid does…”
• Don’t do “Lessons Learned”!!!
• Do do “Long Hard Look”
– Start
– Stop
– Continue
• “Short Hard Look”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
“Now that’s magic!”
• Most people don’t want to know
how the tricks work…
• Don’t discuss “what we do, how
we do it”
• Stay away from practitioner
jargon
• Focus on business outcomes
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Techniques for identifying information
requirements,
Sponsored by Johann Wolfgang Von Goethe
“An intelligent man finds
almost everything
ridiculous.”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Five Whiskies in a Hotel
• What data/information inputs? What outputs? What
tests/measures/criteria will be applied to confirm whether
the data is fit for purpose or not?
• Why do you want it? (What outcomes do you hope to
achieve? Does the data being requested actually support
those questions & outcomes? Consider
Efficiency/Effectiveness/Risk Mitigation drivers for benefit.
• Who is involved? For whom is the information? Who has
rights to see the data? Who is it being provided by? Who is
ultimately accountable for the data - both contents and
definitions? Consider multiple stakeholder groups?
• When is the information required? When is it first required?
Frequency? Trigger events?
• Where is the data to reside? Where is it originating from?
Where is it going to?
• How will it be shared? How will the mechanisms/methods
work to
collect/collate/integrate/store/disseminate/access/archive
the data? How should it be structured & formatted?
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Categorise the comments
• Functions
• Stakeholders
• Inhibitors
• Enablers
• Constraints
• Considerations
• Source systems
• Data sets
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Example Dialectic
• ADD: “What does your team do?”
• Workshop Victim Participant #1: “Well, we’re
responsible for reconciling the customer account
balances with the individual transactions.”
• ADD: And why do you want to do that?
• Workshop Victim Participant #2: “We think
there’s a discrepancy in the warehouse stock
balances, compared with what’s been shipped to
customers. The sales guys keep their own
database of customer contracts and orders and
Jim’s already given us dump of the data, while
finance run the accounts receivables process. But
Sally the Accounts Clerk doesn’t let the numbers
out under any circumstances, so basically we’re
screwed.” “The beginning of wisdom
is the definition of terms”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Example Dialectic: Analysis
• Functions: Sales Processing, Contract
Management, Order Fulfilment, Stock
Management, Accounts Receivable.
• Stakeholders: Warehouse team, Sales team
(Jim), Finance team.
• Inhibitors: Finance don’t collaborate.
• Enablers: Jim is helpful.
• Source Systems: Stock System, Customer
Database, Order Management, Finance System.
• Data Sets: Orders (Quantity & Price by
Customer, by Salesman, by Stock Item),
Dispatches (Quantity & Price by Customer, by
Salesman, by Warehouse Clerk, by Stock Item),
Financial Transactions (Value by Customer, by
Order Ref)
“A good decision is based
on knowledge and not on
numbers.”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Information Asset Management is about mapping
information assets to business outcomes
Owners
AssetManagement
Tools
Governance
Admin
Experts
User Community
Information
Asset
Steward
OwnersOwners
The “Information Asset Community”
Directives
WMSAIRS
Example High Level Data Systems & Flows
Version 1.0
General
Mandatory
Core
Corporate Support
Automatic One-way Relationship
Automatic Two-way Relationship
Manual One-way Relationship
Manual Two-way Relationship
External
External System
AIRSInterchange
AUSSAR
AME
Cyber Exams
ATSL
CLIC
SDR
Publishing System
GMEL
TRIM AD
FRLI
Web Control Mgmt
System (WCMS)
FMIS
STI
ESIR
Comweb
ATO Business
Portal
Inventory Mgmt
System (IMS)
EPK
ComBIZ Online
ProMaster
FCAT
CalumoCBMS(DoFD)
COMCARE
Thomas Logistics
SM7
HRMS
HRFlex
DTAR/OTAR
APEX
AOD Audit
AOD Case Mgmt
SystemTimelog
eRooms
Symbion Health System
API Upload System
ChangePoint Testing System
MRS
ASSP
AWS
AFD
ASIR
ADMS
TrackerATOG Job Register
AOC Surveys
Industry PaymentsCompensation Payments
Financial Actuals
Financial Actuals
Employee Expenses / Adjustment
Journals
Salary Payments
Cash Payments
Payroll (Salary)
Cash Payments / Organisation Info
Human Resources Finance
Surveillance/Audit/Reporting/Tracking
Workflow and Online Collaboration/Service Delivery
Service Delivery
Service Delivery
Service Delivery, HR & Finance, Agreements, Permissions, Aerodromes, Participants, Aircraft
Medical Examinations
Medical Exams
Surveillance /Audits/ Reporting/ Tracking
Alcohol and Other Drugs
Surveys / Surveillance
Events/Occurrences, Aircraft, Aerodromes
Surveys/ Certifications
Examinations
Work Orders
Surveillance /Audits
Aerodromes
Aircraft
Events/Occurrences, Aircraft, Aerodromes
Defects/Events/Occurrences, Aircraft, Aerodromes
Exemptions Database
Alternative Means of
Compliance (AMOC)
Exemptions
AMOC / Exemptions
Human Resources – Flex
Time
Human Resources - Travel
Physical Inventory
Audit Data
workflow / service delivery
workflow / service delivery
Contacts – Ind, Org’s Contacts – Ind, Org’s
ExaminationsMedical
Examinations
Search and Rescue
Surveys
Human Resources – Time
Aircraft Equipment Finances
MMELBaseline/Minimal Equipment
Medical
PAWS
Retain
Details of Operators
Incidents
Applications / Permissions
Trending
Workflow
MAAT
Permissions / Change of Status
Permissions / Change of Status
Service Log
Alternative Means of Compliance (AMOC)
Dangerous Goods
Dangerous Goods
Content
Inventory
FTTO
FTNS
Individual Flight Data
Organisational Flight Data
Human Resources – Time
Aircraft
Individuals/ARNS
Payments
HR - TimeCash Receipts
Reconcile Invoice against
Flown Hours
Surveys / Surveillance
Enterprise Data Warehouse
CASA Internet
Airports
Landings/ Take Offs
Data
Mandatory
Core
Corporate Support
External
Business Process
Surveillance/Audit/
Reporting/Tracking
Bank Data File
PAYG payments, Salary
payments, and Superannuation
payments.
External Superannuation
Companies
Cash Payments
Superannuation Contributions
Suppliers
Remittance Advice
300+ Access Databases
Contacts
Airspace
Organisational
Human Resources
Aircraft
Permissions Info Asset Register
(inventory) System Interfaces map
“Science is organized
knowledge. Wisdom is
organized life.”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Actionable Business Outcomes,
Sponsored by Thomas Edison
“The value of an idea
lies in the using of it.”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Actionability = Value-add
• Look for self-assigned actions
e.g.
Workshop Victim Participant #1:
“I could go and speak to the
Financial Controller about getting
access to the finance data. He’s
more amenable to working
together than Sally, who just does
what she’s told.”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Enabling method: CRUD Matrix
• Create
• Read
• Update/Use
• Delete
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Enabling method: Information Lifecycle Map
Time table
Handbook
Archibus
UniForumCompletion rates for
HDA
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Enabling Method: DQ profiling
• Data Quality Profiling and root-
cause analysis
• Identify issues & patterns
• Initiation activity as part of all
data warehouse, master data &
application migration projects
“Whoever is careless with
the truth in small matters
cannot be trusted with
important matters.” Common Data Quality processes and collaboration workspace
Collect Validate Protect Monitor
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Pay a visit to Whitefellah Burrows…
• Know what you’re
looking for (and why)
• Dig in the right areas
• Sift through the mullock
• Be prepared for some
hard digging
• Be careful or you might
get hurt!
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Conclusions & final thoughts,
Sponsored by Terry Pratchett
“It’s still magic even if
you know how it’s
done.”
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Resources: info requirements templates
• http://informationaction.blogspot.co.uk/p/resou
rces-and-downloads.html • 01. Framework Overview
• 02. Information Solution Outline & High Level Requirements Template
• 03. Business Information Requirements Template
• 04. Logical Data Definition Template
• 05. Physical Data Specification Template
• 06. Transformation Logic Template (Source to TargetMapping)
• 07. Analytics & Reporting Requirements Template
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Intellectual curiosity
Skeptical scrutiny
Critical thinking http://www.informationaction.blogspot.com.au/
http://blogs.gartner.com/alan-duncan/
@Alan_D_Duncan
http://www.linkedin.com/in/alandduncan