sample brown paper findings release improvement project

Sample Brown Paper Findings

Release Improvement Project

- 2 -

RELEASE IMPROVEMENT PROJECT A&D: BROWN PAPER FINDINGS

Brown Paper Objectives

• Understand the “As-Is” release processes–Stay at the “forest” level–Findings not process

–How things usually happen, not all the what-if loops

• Provide a medium to solicit process-user input

• Begin to build enthusiasm for the change process

- 3 -


Brown Paper Approach

• Built six brown papers:–C30 Release – SRC Operations

–D10 – Defect Repair

–D20 – Release Planning*

• Joint team building sessions with xxxxxx information holders:–SRPM focused

–Multifunctional resources

• Held four brown paper faires to solicit user input from Developers, QA, PUBs, Product Management, etc.

* Findings incorporated into IPM Process panel set

- 4 -


Wide Spread TSG Support and Enthusiasm for the Brown Paper Process Is Exemplified by the Large Participation

Brown Paper

C30

D10

D20

SRC Operations

Defect Repair

SRPM Planning

At least 100 people provided hands-on input to the brown paper writing over 1,000 comments.

Participants

C30, D10, D20Findings

- 6 -


Strengths

• Shortfalls in the “process” are compensated for by great people-effort

• Release core team is a good concept

• Weekly submittal windows help drive quality into the system – social system allows people to work from same starting point

• Periodic Q&A sessions around software release have been helpful

• D20's upfront architecture work seems to have built a solid base so far

- 7 -


There Is No One Set Process to Release Software

• “Development is starving for senior management to impose rules, guidelines and well defined processes.”

• “Processes are complicated because all development organization don't follow the same requirements/guidelines.”

Releases differ by milestones, rules, and work steps

D10

D20

C30

- 8 -


To Utilize Multiple Release Vehicles Means Becoming an “Expert” for Each Release

• “I'm releasing in C30 as long as possible to avoid the D10 rules”

• “I didn't release in D10 because I would have had to create new Softdocs.”

• “D20 is consciously built to not look like D10.”

D10“Release Expert”

D20“Release Expert”

C30“Release Expert”

Development avoids multiple threads as much as possible.

- 9 -


Beating the System Becomes the Norm

• “I bypass weekly test cycle with ‘under the table’ submits to QA.”

• “Too many rules have evolved. So I release with work arounds.”

• “If you are endorsed by the right people you can always get your code in.”

• “Techs network informally to get different components for a certain release.”

Cecil Counting His Profits

- 10 -


Nothing Is Every Truly “Frozen”

• “…Code freeze often turns to code slushy.”

• TPR freezing seems arbitrary depending on release:

–Freezing too early can cause noncritical problems to never get fixed

What we do:

• Unfreeze content

• Unfreeze functionality

• Inflate TPR severity to SEV 3 and repair

• Slip deadlines

What we think:

• Freeze content

• Freeze functionality

• Freeze SEV 0 + 1 TPRs

• Freeze schedulesSlushy

- 11 -


Limited Schedule Credibility Has Down-Stream Impact

• Numerous Softdoc iterations/compressed CD Rom Production

• PUBs rework/short-manual lead times

• “Content slush leads to document rewritings”

• SUT rebuilds/refreshes

Slushy

- 12 -


Blind Hand-Offs from Development to QA Lead to Rework

• “QA tests should be run on development systems by developers.”

• “Too much integration testing in QA.”

• “We need division level integration testing processes.”

• “Development provides QA with unit test plans and QA provides Development with system test plans… Differentiation is unclear.”

• “Need visibility of QA's ability and readiness to perform testing; need criteria and metrics for start of QA.”

Weekly Development/QA Magtape Frisbee Toss

- 13 -


Weekly Submittal Process Is Characterized as Inefficient and Schedule Driven

• “Windows were first introduced as a bandaid…not meant to become part of the process.”

• “Developers chase Windows more than they pay attention to quality.”

• “With Window system, developers are out of business at least 28% of the time.”

• “Weekly SUTs are cut, no matter what, to hit the schedule; we shouldn't cut SUTs if code doesn't run or can't be installed.”

• “We need to stop sacrificing quality for schedule.”

• “It's hard to fix, build, test, and submit code in a week.”

• “Windows should be controlled on the SRC side:

–Development could submit anytime and SRC could coordinate the code.”

- 14 -


Implementers Have Limited Input to the Planning Process Resulting in Noncredible Schedules…

TANDEM Calendar November

Sunday Monday Tuesday Wednesday Thursday Friday Saturday3

10

17

24

4

11

18

25

5

12

19

26

6

13

20

27

7

14

21

28

1

8

15

22

29

2

93

16

23

30

Release PlanDecision Makers

• “Why request quarterly input when releases are less frequent than quarterly.”

• “No rewards or penalties for making or missing schedules.”

• “Shouldn't commit to plan until firm design plans are in place.”

• “Program and schedule plans were severely handicapped by lack of QA staff early on the program.”

Implementers

…But are responsible for deliverables.

SRPMDeveloper

QAPUBs

DeveloperManager

ProductManagement

- 15 -


Core Team Roles and Responsibilities Are Unclear

• “Core teams lack consistent behavior, rules… they need training.”

• “Release content is being determined by non-technical people.”

• “Core team is too far removed to determine what should and shouldn't be allowed for submittal.”

- 16 -


D20 Seems Stuck in a Vicious Cycle

“I can't give you a good SUT until you give me a

good product”

“I can't build a good product without a good

SUT”

• 23 weeks of iterations…still, not reached “one-good-SUT” milestone

• Different opinions exist for TPR filings

–“Need to marry old and new school thought”

• “We never slow down enough to get to functional milestones.”

SRC/SRPMDevelopment

SRC Findings

- 18 -


Strengths

• People

- "SRC Staff are always willing to help"

- "SRC people exhibit a strong desire to work through a difficult process"

• BIT testing procedure

- "BIT testing process is excellent at finding "dead on arrival" SUTs"

• - "Frequent submittal windows are helpful"

- 19 -


Complex Release Procedures and Rules Cause Confusion and Frustration

Much energy is expended on low value added activities- statusing , manual validation

- 20 -


Lack of Flexibility in Release Rules Results in an Inabilityto Distinguish Between Major and Minor Problems

Typos in SOFTDOCS are treated the same as code failure

- 21 -


Limited Screening/Testing by Development Leads to Errors Being Caught Later in the Release Cycle

Costs to rectify errors increase the further downstream in the process . . . SRC is the developers "watchdog"

- 22 -


Antiquated Tools Enslave Rather Than Empower the SRC Process

Most release tools are at least 5 years old and no longer support s xxxxxx's current and future needs

- 23 -


Too Many SOFTDOCS "Gods" Lead to Confusing Rules

No one "owns" SOFTDOCS. As a result:

• There is a lack of standardization and documentation

• Users use SOFTDOCS to fulfill a variety of information needs:

•• Dependency identification

•• Internal documentation

•• Installation Procedures

SOFTDOCS serves many masters . . . none very well

Release Planning Findings

- 25 -


Strengths of Release Planning

• “At least we're planning now. We used to not do it.”

• “Market requirements and product objectives process is designed to prioritize market requirements and translate to objectives. The goal is good.”

• “Customer Advisory Boards are a great idea. Let's do more. Let's involve customers in our future.”

• “Quarterly updates to development plans are helpful. I refer to this a lot.”

• “The strategy briefings are a great improvement in communication.”

• “For what does get into the formal process (commitments), we do well.”

• “Planning for software tied to hardware is better than for other software.”

• “…xxxxxx's top 10 planning priorities are useful.”

• “The iterative steps in the process provide ‘good visibility’ to the plan.”

- 26 -


Customer Input Is Only Weakly Tied to Release Planning

RELEASE PLAN November

Sunday Monday Tuesday Wednesday Thursday Friday Saturday3

10

17

24

4

11

18

25

5

12

19

26

6

13

20

27

7

14

21

28

1

8

15

22

29

2

93

16

23

30

“xxxxxx Needs”

“Customer Needs”

• “Most of this (customer input) is missing. We are problem driven.”

• “All xxxxxx groups must return their focus to customer needs — not the political BS of internal affairs.”

• “Need more direct interaction with customers.”

The planning process is largely internal, not customer driven.

- 27 -


Planning Doesn't Adjust to Mid-Year Changes Well

• “Customers need change. The process doesn't address this well.”

• “We need a process to update resource plans, not just steal resources.”

• “We don't do enough to see if we funded a project properly. We borrowed resources from other projects.”

• “Development resources are often diverted from projects, making schedule estimating/committing difficult.”

Meeting schedules is difficult when resources are “stolen” mid-project.

Change

FINISH

Change

Change

Change

PLAN

- 28 -


Many Employees Feel the Three Year Planning Process Is Ineffective

• “The second year of a three year plan is often changed, and the third year is never right.”

• “The last project I was on, the three year plan was off by 2 1/2 years.”

• “Three year plan numbers are meaningless for a lot of projects—made up. Evaluating cost/schedules from three year plans isn't basing designs on reality.”

• “My three year lasted six months.”

Planning beyond one year has little value to employees.

3rd Year Plan

2nd Year Plan

1st Year Plan

- 29 -


Development Planning Is Ineffective

• “We don't' teach project management, so people don't know how to make accurate plans.”

• “We lack both skills and historical data to estimate accurately.”

• “We don't know how to estimate, so we don't know how to make schedules or meet them, so plans are meaningless.”

• “We never do post-mortems on project budgets.”

Planning capabilities need to be developed.

January February March

April May June

July August September

October November December

RELEASE RELEASE

RELEASE June

RELEASE

- 30 -


Frequently of Release Is Red Issue for Development

• “Release bottleneck: New product or enhancement is complete but can't be shipped except as an IPM due to all or nothing Guardian releases.”

• “Customers won't wait forever. Plan for frequent, incremental improvement.”

• “The world changes more frequently than once a quarter.”

• “Release process cycle time (too long) leads to changes in assumptions, which leads to development rework to “fit” with new assumptions.”

Products are waiting at the station, but the train isn't ready to board.

STOP

Wait For Next

Release

- 31 -


Release Process Information Is Not Shared

• “We need to set up an information directory…”for plans

product dependenciescustomer commitments

“…and make it available to all.”

• Too many private distribution spreadsheets and documents with limited distribution

• “Need to send automatic reminders to person assigned…”

• “FREs go to Product Management. Once that happens, they are never heard from again.”

Required information is not readily available.

SNAX has nodependencies

SNAX depends on spooler

- 32 -


Release Rules and Decisions Are Unclear

• “Release rules are not consistent and change frequently. We need to know the rules.”

• “Decision criteria are not understood, justified, and communicated.”

• Role of core team and middle/executive management questioned:

–“Who are these people?”

–“They do not understand…”

…made by a few, but impact many.

- 33 -


Functional Plans Are Not Integrated

• “QA and Development should coordinate their plans before submitting to release planning.”

• “xxxxxx should have a Master Plan, from which the Release plan can be derived.”

• “Publications plan not well integrated within release plan.”

• “Publications plan not included in 3-year Development plan.”

Release plan does not drive all work activity.

My Plan My Plan

What's theplan?

- 34 -


Communication of Priorities and Commitments Is Inadequate

• “Only about 30% of the commitments get into the process.”

• “Don't appropriately differentiate between major projects and little stuff.”

• “Who analyzes customer commitment requests? Product Management doesn't, and Development is isolated from customers.”

• “Establish and stick to priority commitments. Don't tweak so frequently.”

• “There are still many customer commitments which do not get viability. Commitments are made without enough thought.”

I'll write down that

commitment tomorrow

We'll have that to you next Tuesday

sample brown paper findings release improvement project

Documents

brown paper process

release improvement

software release

didnt release

certain release

sample brown paper findings

brown paper faires

brown paper writing