software project risk assessments: it’s time to determine how to
TRANSCRIPT
Software Project Risk Assessments: It’s Time To Determine How To Get Back On Track
Dan Galorath, Subject Matter Expert
Donald Reifer, Subject Matter Expert
1
Experienced Presenters
Donald Reifer – Leader in the field with 40+ years of experience in software engineering & management in both government and industry.
Dan Galorath – Leader with 40+ years software engineering, management, product management
2
Learning Objectives (ddg)
• Teach how to conduct a risk assessment • What is it• Why is it done• Who performs it• Why should you
schedule one?• How to prioritize risks
based on potential impacts
• Who should consider conducting risk assessment?
• Answer questions
3
• Avoid Future Costs• Shorten Schedules• Get Back on Track
Take Aways From Webinar (ddg)
4
• Identification of roles and responsibilities
• Typical timelines• Planning checklists
• Before the assessment• During the assessment• After the assessment
• Emphasis on dos and don’ts
• Reference listBeware of Dragons
Risk Assessments Help Make Projects Successful (ddg)
• Identify issues that can impede delivery of an acceptable product on-time & within budget
• Often done at key milestones or to bail out a project in trouble
• Can be done internally, by the customer or by an outside, independent assessor
5
“When you find yourself in a hole, stop digging”…Will Rogers
Software Risk Assessment Focus On Issues and Resolution (ddg)
• Focus on identifying issues that matter & actions to resolve them• Emphasis on biggest
impacts on success
• Get an unbiased, expert, third-party opinion effort to complete• Often from firm that has
seen similar results and can prescribe actions that work
6
“Action is the foundational key to all success.”
…Pablo Picasso
Risk Assessments Can be Strategic or Situational (ddg)
At Strategic Milestones• Third party look helps
you stay on the path to success• Identify issues that
you should be worried about
• Provide situational assessment from both the customer & competitive viewpoints
• Show you are serious about succeeding
When You Are In Trouble• Perform risk
assessment before customer comes in and performs one• Shows customer you
are serious about correction
• Limits the damage a customer assessment can do especially when people have serious work to get done
7
Delusions of Success: How Optimism Undermines Executives' Decisions (Source: HBR) (ddg)
• Problem:
Humans seem hardwired to be optimists
• Optimism from cognitive biases & organizational pressures• Exaggerate talents & degree of control
• Attribute negative consequences to external factors
• Anchoring (relying too heavily on one piece of information) magnifies optimism• Most pronounced for new initiativesSolution: Temper with “outside view”
Don’t remove optimism, but balance optimism & realism
Unbiased Assessment is Most Valuable (ddg)
Who IssuesInternal Often see things through rose
colored glasses
Difficult to “tell it like it is”Customer Team Tell you exactly what they think
is wrong
Rarely tell you how to fix things: that’s your job
Independent Expert Team Quick, hard-hitting and relatively inexpensive
Hired to identify issues and tell you what to do about them
Provide insight into what has worked for others within your industry and/or marketplace
© 2012 Copyright Galorath Incorporated 9
Types of Assessments (djr)
• Quick-Look Assessment• Narrower focus on six
activities
• Look at drivers that impact cost and schedule
• Focus is on issues and their causes
• Takes a team of 2 about a week to complete
• Products are actions that need to be taken to get back on track
• Full Assessment• Broad focus on
eighteen activities
• Look at drivers that impact cost and schedule
• Focus is on issues and their “root causes”
• Takes a team of 3 about a month to complete
• Products are actions that need to be taken to get back on track
© 2012 Copyright Galorath Incorporated 10
Full Assessment Root Cause Analysis (ddg)
• When risk assessment finds an issue…• Probe: Ask why, again and again, until you ferret out
the root cause (often the 5 whys)
• Recommend correction of the root cause, not the symptoms
• E.g. Why is the software late…
• Because requirements changed
• Because customer changed mind
• Because we didn’t provide vision…
• Because we were rushed
• Because the customer wouldn’t take time up front
• And we didn’t account for that in our estimates
• Keep going until answer is “don’t know” or don’t care
Full Analysis: Root Cause Analysis (ddg)
Projectfloundering
Unstablerequirements
Caused By
NoSchedule
Norequirements
baseline
No estimationof change impact
Caused By
Caused By Etc.
Caused By
Evidence goes here
Evidence goes here
Evidence goes here
Evidence goes here
Evidence goes here
Example Process Area Assessment Details (ddg)
Assessment is valuable….. Root cause analysis and Recommended Solutions Are Priceless
List of Roles and Responsibilities (ddg)
• Enterprise-Level Risk Assessment• Lead – Independent
assessor (from outside of the enterprise)
• Team Members –subject matter experts in areas being assessed
• Patrons – senior management
• Participants - action assignees and those who contributed
• Software Project Risk Assessment• Lead – Independent
assessor (from outside group being assessed)
• Team Members –subject matter experts in areas being assessed
• Patrons – sponsors of assessment
• Participants –action assignees and those who contributed
14
Typical Timelines (djr)
Full AssessmentActivity Timeline
Form team - 6 weeksSend out questionnaire - 4 weeksReceive informationpackage
- 2 weeks
Conduct assessmentBrief findings to all contributors and management at end
0(Full week)
Conduct impact analysis
+ 2 weeks
Executive briefing + 3 weeksPublish final report + 4 weeks
15
Quick-Look AssessmentActivity Timeline
Form team - 4 weeksSend out questionnaire
- 3 weeks
Receive informationpackage
- 1 weeks
Conduct assessmentBrief findings to all contributors and management at end
0(2 to 3 days)
Publish final report + 1 week
Information packages differ
“Before” Checklist (djr)
Before the assessment• Team and experienced leader assigned• Information package requested to
include:• Organization structure including bios of
key personnel• Organization or project background
including history • Questionnaire with explanatory notes
• Kick-off teleconference scheduled and held• Ensure the goals of assessment are
explained• Expectations need to be set• Schedules need to be agreed to
• Review information when received• Be up to speed and ready to start as
soon as you arrive at assessment
16
“Be prepared and be honest.”
…John Wooden
Typical Schedules (Full) (djr)
• Full Assessment• Intense
• Comprehensive• 18 activities
• Tell it like it is
• Covers• Management
• Technical
• Support
• Diagnostic
• Remedial
Day Time Activity / Process Area1 AM Introductions
1. Program Management2. Program and Build Planning3. Configuration Management
PM 4. Systems Engineering5. Testing and Test Automation6. Quality Assurance7. Delivery and Deployment
2 AM 8. Risk Management9. Cost & Schedule Controls10. Metrics and Measurement
17
Typical Schedules (Full) (djr)
• Full Assessment• Gather facts
• Confirm them
• Identify issues
• Identify actions
• Brief management• Findings
• Conclusions
• Recommendations
• Help you to help yourself
Day Time Activity / Process Area2 PM 11. Earned Value Management
12. Supplier Management13. Integrated Product Teams14. Human ResourcesManagement
3 AM 15. Requirements Management16. Interface Management17. Continuous Integration / Agile Methods
PM 18. Special Topics / SecurityWrap-up and Out-briefExecutive Briefing
18
“During” Checklist (djr)
During Assessment• Gather as much data as possible in
advance to get up to speed before assessment• Questionnaires provided for that
purpose• List of information for package and
checklists• Conduct interviews on-site at all levels• Provide out-briefings to all contributors• Be perceived as fair and unbiased• Further quantify issue impacts using
cost models either during or after assessment
• Be sure to answer management’s questions
• Summarize findings and conclusions• Provide actionable recommendations
19
“Any assessment is a perpetual work in progress.”
…Linda Suske
What Are The Typical Findings? (djr)
• Findings – Managerial• Behind schedule
• Under-spending
• Over-spending
• Supplier mgt
• Risk management processes not working
Original schedule was impossible to start, devise new plan
Staffing up harder and taking longer than anticipated
Trying to catch up by adding staff, sometimes impossible and other times possible
Licensing & subcontracting
Revise process and get senior management involved
20
More Findings (djr)
• Findings – Technical• Systems engineering
behind
• Integrated product teams not working
• Requirements volatile and gold-plating
• Architecture improperly specified
• Agile metrics
• Findings – Support• QA just an exercise,
not a passion
Too much to do, too little staff - rescope
Improperly done, fix
Stabilize and simplify
Need to portray three views of architecture
Insight into progress
Rebuild team and give them a mission
21
Typical Outputs (djr)
• Outputs• List of issues prioritized by
impact• Top five to ten issues and
recommended actions• Justifications for taking
actions in dollars and cents terms
• Scorecard that is color coded to highlight findings
• Report that can be used by response team to better understand findings, conclusions and recommendations
22
“It is surprising how few of our risk assessment clients are surprised by our findings.”
… Dan Galorath… Don Reifer
“After” Checklist (djr)
After Assessment
• Report it like it is, but do so kindly• Your job is to tell them what you
found and what to do about it
• Do not betray confidences
• Remember, if there are issues, there must be solutions
• Make sure terms of the engagement are satisfied before you finish
• Follow up and follow through on what you promised during the assessment
• Reassess after a reasonable time period
23
“It is easy to pass the buck. It is a lot harder tomake a difference.”
…Tom Brokaw
Typical ROI (djr)
• When you find and fix major problems early, ROI can be substantial• Rework reductions can save
thousands• Finding the root causes – that’s
where outside experts add value• Rework caused by assuming
you can reuse legacy code when in fact you can’t
• Legacy code is 4 times more error-prone than normal, has secondary impacts on overall product quality
• Sometimes better to either use COTS or build a simple replacement for the legacy code
24
“Quality is not an act, it is a habit.”
...Aristotle
The Galorath Process (ddg)
• Standard Repeatable Process• Evolved based on “Best Practice” • Process uses many models and miscellaneous
specialized evaluation tools to quantify results • We collect and display results via visual
scorecard (stoplights)• When possible, we quantify results using
models & metrics
• Issues-Based, Action-Oriented Results
• Process generates actions that address issues identified and can be put to work immediately
25
Process Looks At The Factors That Drive Cost and Schedule (ddg)
Cost drivers- Teamwork
- Staff skills
- Requirements volatility
- Product complexity
- Degree of automation
……..
• Most assessments focus on issues
• In contrast, the Galorath process focuses / quantifies issues that count
• They drive costs
• They influence schedules
• They impact your ability to deliver
• They relate to your competitive advantages and disadvantages
• They are things that you can do something about
26
Summary and Conclusions (ddg)
• Discussed software project risk assessments • What they are, why they are done,
who performs them and why you should schedule one
• The process that Galorath uses to conduct them
• Who should consider conducting one
• Are there any questions?
27
References
• Books• Galorath and Evans, Software Sizing,
Estimation and Risk Management, Auerbach Publishers, 2006.
• Pandian, Applied Software Risk Management: A Guide for Software Project Managers, Auerbach Publishers, 2006.
• Reifer, Making the Software Business Case: Improvement by the Numbers, Addison-Wesley, 2001…
• Websites• Galorath Incorporated www.galorath.com• Institute of Risk Management www.theirm.org• Project Management Institute www.pmi.org• … Many others
28
Contact Information
• For more information, contact:
Brian Glauser, VP
Galorath Incorporated
222 N. Sepulveda Blvd.
Suite 1700
El Segundo CA 90245
310-414-3222 Ext 231
• To schedule a risk assessment, contact your Galorath representative
29