the one question you must never ask!" (information requirements gathering for analytic...

29
Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan Information Strategy | Data Governance | Analytics | Better Business Outcomes Information requirements gathering for data projects, Sponsored by W. Edwards Deming The one question you must never ask! If you do not know how to ask the right question, you discover nothing.”

Upload: alan-d-duncan

Post on 25-Jun-2015

317 views

Category:

Data & Analytics


3 download

DESCRIPTION

Presentation from 2014 International Data Quality Summit (www.idqsummit.org, Twitter hashtag #IDQS14). Techniques for business analysts and data scientists to facilitate better requirements gathering in data and analytic projects.

TRANSCRIPT

Page 1: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Information requirements gathering for data projects,

Sponsored by W. Edwards Deming

The one question you must never ask!

“If you do not know how

to ask the right question,

you discover nothing.”

Page 2: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Agenda

• Challenges of info requirements gathering

• Methods for engaging with data stakeholders

• Techniques for identifying info requirements

• Establishing actionable business outcomes

Page 3: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

A bit about me....

• NEW! Research Director, Gartner

• Advisory board member, QFire Software

• 22 years Information Strategy, Data Governance,

Analytics & Business Consulting

– Director of Data Governance at UNSW

– EDS, KPMG, CPW, Acuma, Pelion, SMS

– Scottish Power, United Distillers, O2, Astra Zeneca,

Carphone Warehouse, Vodafone, Riyad Bank

– Commonwealth Bank, NSW Roads & Maritime

Services, Centrelink, OATSIH, NSW Family &

Community Services, CASA, AMSA, FaHCSIA, DAFF,

Navy…

• Information-Management.com “Top 12 on Twitter”

• Best supporting Actor, 2005 Barnet Drama Festival

Page 4: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Challenges for

information requirements gathering,

Sponsored by Sir Arthur Conan Doyle

“Data! Data! Data! I can’t

make bricks without clay!” (Sherlock Holmes,

The Adventure of the Copper Beeches)

Page 5: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

“What’s the one

question that you

should never ask?”

Page 6: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

“What information do

you want?!”

Page 7: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Page 8: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Challenges for Information Requirements gathering…

Lack of knowledge about new

information-processing technologies

People don’t know what they want!

Unmet & changing demands

Competition between teams

Page 9: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

…plus second-guessing future needs.

Page 10: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Methods for engaging data stakeholders,

Sponsored by Brene Brown

“Maybe stories are just data

with a soul.”

Page 11: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Information Workshop Agenda

• Context setting

• Business Functions

• Challenges

• Opportunities

• Desired Actions

Page 12: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Don’t just facilitate - provoke!

• “What would happen if…X?”

• “Have you thought about…Y?”

• “Why do you do/want…Z?”

Page 13: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

“Stupid is as stupid does…”

• Don’t do “Lessons Learned”!!!

• Do do “Long Hard Look”

– Start

– Stop

– Continue

• “Short Hard Look”

Page 14: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

“Now that’s magic!”

• Most people don’t want to know

how the tricks work…

• Don’t discuss “what we do, how

we do it”

• Stay away from practitioner

jargon

• Focus on business outcomes

Page 15: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Techniques for identifying information

requirements,

Sponsored by Johann Wolfgang Von Goethe

“An intelligent man finds

almost everything

ridiculous.”

Page 16: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Five Whiskies in a Hotel

• What data/information inputs? What outputs? What

tests/measures/criteria will be applied to confirm whether

the data is fit for purpose or not?

• Why do you want it? (What outcomes do you hope to

achieve? Does the data being requested actually support

those questions & outcomes? Consider

Efficiency/Effectiveness/Risk Mitigation drivers for benefit.

• Who is involved? For whom is the information? Who has

rights to see the data? Who is it being provided by? Who is

ultimately accountable for the data - both contents and

definitions? Consider multiple stakeholder groups?

• When is the information required? When is it first required?

Frequency? Trigger events?

• Where is the data to reside? Where is it originating from?

Where is it going to?

• How will it be shared? How will the mechanisms/methods

work to

collect/collate/integrate/store/disseminate/access/archive

the data? How should it be structured & formatted?

Page 17: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Categorise the comments

• Functions

• Stakeholders

• Inhibitors

• Enablers

• Constraints

• Considerations

• Source systems

• Data sets

Page 18: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Example Dialectic

• ADD: “What does your team do?”

• Workshop Victim Participant #1: “Well, we’re

responsible for reconciling the customer account

balances with the individual transactions.”

• ADD: And why do you want to do that?

• Workshop Victim Participant #2: “We think

there’s a discrepancy in the warehouse stock

balances, compared with what’s been shipped to

customers. The sales guys keep their own

database of customer contracts and orders and

Jim’s already given us dump of the data, while

finance run the accounts receivables process. But

Sally the Accounts Clerk doesn’t let the numbers

out under any circumstances, so basically we’re

screwed.” “The beginning of wisdom

is the definition of terms”

Page 19: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Example Dialectic: Analysis

• Functions: Sales Processing, Contract

Management, Order Fulfilment, Stock

Management, Accounts Receivable.

• Stakeholders: Warehouse team, Sales team

(Jim), Finance team.

• Inhibitors: Finance don’t collaborate.

• Enablers: Jim is helpful.

• Source Systems: Stock System, Customer

Database, Order Management, Finance System.

• Data Sets: Orders (Quantity & Price by

Customer, by Salesman, by Stock Item),

Dispatches (Quantity & Price by Customer, by

Salesman, by Warehouse Clerk, by Stock Item),

Financial Transactions (Value by Customer, by

Order Ref)

“A good decision is based

on knowledge and not on

numbers.”

Page 20: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Information Asset Management is about mapping

information assets to business outcomes

Owners

AssetManagement

Tools

Governance

Admin

Experts

User Community

Information

Asset

Steward

OwnersOwners

The “Information Asset Community”

Directives

WMSAIRS

Example High Level Data Systems & Flows

Version 1.0

General

Mandatory

Core

Corporate Support

Automatic One-way Relationship

Automatic Two-way Relationship

Manual One-way Relationship

Manual Two-way Relationship

External

External System

AIRSInterchange

AUSSAR

AME

Cyber Exams

ATSL

CLIC

SDR

Publishing System

GMEL

TRIM AD

FRLI

Web Control Mgmt

System (WCMS)

FMIS

STI

ESIR

Comweb

ATO Business

Portal

Inventory Mgmt

System (IMS)

EPK

ComBIZ Online

ProMaster

FCAT

CalumoCBMS(DoFD)

COMCARE

Thomas Logistics

SM7

HRMS

HRFlex

DTAR/OTAR

APEX

AOD Audit

AOD Case Mgmt

SystemTimelog

eRooms

Symbion Health System

API Upload System

ChangePoint Testing System

MRS

ASSP

AWS

AFD

ASIR

ADMS

TrackerATOG Job Register

AOC Surveys

Industry PaymentsCompensation Payments

Financial Actuals

Financial Actuals

Employee Expenses / Adjustment

Journals

Salary Payments

Cash Payments

Payroll (Salary)

Cash Payments / Organisation Info

Human Resources Finance

Surveillance/Audit/Reporting/Tracking

Workflow and Online Collaboration/Service Delivery

Service Delivery

Service Delivery

Service Delivery, HR & Finance, Agreements, Permissions, Aerodromes, Participants, Aircraft

Medical Examinations

Medical Exams

Surveillance /Audits/ Reporting/ Tracking

Alcohol and Other Drugs

Surveys / Surveillance

Events/Occurrences, Aircraft, Aerodromes

Surveys/ Certifications

Examinations

Work Orders

Surveillance /Audits

Aerodromes

Aircraft

Events/Occurrences, Aircraft, Aerodromes

Defects/Events/Occurrences, Aircraft, Aerodromes

Exemptions Database

Alternative Means of

Compliance (AMOC)

Exemptions

AMOC / Exemptions

Human Resources – Flex

Time

Human Resources - Travel

Physical Inventory

Audit Data

workflow / service delivery

workflow / service delivery

Contacts – Ind, Org’s Contacts – Ind, Org’s

ExaminationsMedical

Examinations

Search and Rescue

Surveys

Human Resources – Time

Aircraft Equipment Finances

MMELBaseline/Minimal Equipment

Medical

PAWS

Retain

Details of Operators

Incidents

Applications / Permissions

Trending

Workflow

MAAT

Permissions / Change of Status

Permissions / Change of Status

Service Log

Alternative Means of Compliance (AMOC)

Dangerous Goods

Dangerous Goods

Content

Inventory

FTTO

FTNS

Individual Flight Data

Organisational Flight Data

Human Resources – Time

Aircraft

Individuals/ARNS

Payments

HR - TimeCash Receipts

Reconcile Invoice against

Flown Hours

Surveys / Surveillance

Enterprise Data Warehouse

CASA Internet

Airports

Landings/ Take Offs

Data

Mandatory

Core

Corporate Support

External

Business Process

Surveillance/Audit/

Reporting/Tracking

Bank Data File

PAYG payments, Salary

payments, and Superannuation

payments.

External Superannuation

Companies

Cash Payments

Superannuation Contributions

Suppliers

Remittance Advice

300+ Access Databases

Contacts

Airspace

Organisational

Human Resources

Aircraft

Permissions Info Asset Register

(inventory) System Interfaces map

“Science is organized

knowledge. Wisdom is

organized life.”

Page 21: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Actionable Business Outcomes,

Sponsored by Thomas Edison

“The value of an idea

lies in the using of it.”

Page 22: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Actionability = Value-add

• Look for self-assigned actions

e.g.

Workshop Victim Participant #1:

“I could go and speak to the

Financial Controller about getting

access to the finance data. He’s

more amenable to working

together than Sally, who just does

what she’s told.”

Page 23: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Enabling method: CRUD Matrix

• Create

• Read

• Update/Use

• Delete

Page 24: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Enabling method: Information Lifecycle Map

Time table

Handbook

Archibus

UniForumCompletion rates for

HDA

Page 25: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Enabling Method: DQ profiling

• Data Quality Profiling and root-

cause analysis

• Identify issues & patterns

• Initiation activity as part of all

data warehouse, master data &

application migration projects

“Whoever is careless with

the truth in small matters

cannot be trusted with

important matters.” Common Data Quality processes and collaboration workspace

Collect Validate Protect Monitor

Page 26: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Pay a visit to Whitefellah Burrows…

• Know what you’re

looking for (and why)

• Dig in the right areas

• Sift through the mullock

• Be prepared for some

hard digging

• Be careful or you might

get hurt!

Page 27: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Conclusions & final thoughts,

Sponsored by Terry Pratchett

“It’s still magic even if

you know how it’s

done.”

Page 28: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Resources: info requirements templates

• http://informationaction.blogspot.co.uk/p/resou

rces-and-downloads.html • 01. Framework Overview

• 02. Information Solution Outline & High Level Requirements Template

• 03. Business Information Requirements Template

• 04. Logical Data Definition Template

• 05. Physical Data Specification Template

• 06. Transformation Logic Template (Source to TargetMapping)

• 07. Analytics & Reporting Requirements Template

Page 29: The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)

Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan

Information Strategy | Data Governance | Analytics | Better Business Outcomes

Intellectual curiosity

Skeptical scrutiny

Critical thinking http://www.informationaction.blogspot.com.au/

http://blogs.gartner.com/alan-duncan/

@Alan_D_Duncan

http://www.linkedin.com/in/alandduncan