A Practical Application of theA Practical Application of theNonNon--Advocate ReviewAdvocate Review

BruceBruce NishimeNishime

Over 898,750 Flight Hours!USAF Fleet – 872,885 UK Fleet – 23,085

Over 898,750 Flight Hours!USAF Fleet – 872,885 UK Fleet – 23,085

2

M E E T I N G O U R C O M M I T M E N T S M E E T I N G O U R C O M M I T M E N T S

� Excellent Quality� Ahead of Schedule� On Price� 180 Aircraft Program

�� Excellent QualityExcellent Quality�� Ahead of ScheduleAhead of Schedule�� On PriceOn Price�� 180 Aircraft Program180 Aircraft Program

� 138 USAF Aircraft - 6 Bases� Worldwide Operations� Best Fleet Reliability� 4 UK C-17s Delivered

�� 138 USAF Aircraft138 USAF Aircraft -- 6 Bases6 Bases�� Worldwide OperationsWorldwide Operations�� Best Fleet ReliabilityBest Fleet Reliability�� 4 UK C4 UK C--17s Delivered17s Delivered

CC--17: A High Performance Program17: A High Performance Program

CC--17 Awards17 Awards

2001 ISO9001-2000 /AS9100A Certification

2001 UK MOD SmartAcquisition Award

1994Collier Award

1998 MalcolmBaldrige National

Quality Award

1996 CaliforniaQuality Award

2002 California Awardsfor Performance Excellence

(Gold & Silver)

2003 Missouri StateQuality Award

2002 IW Winner:Top 10 Best Plants

2003 Governor’s Award ForPerformance Excellence

2003 GeorgiaOglethorpe

Award

SEIStandardLevel 5

*CAPE = California Awards for Performance Excellence

MalcolmBaldrige

Win

MalcolmBaldrige

Win

MalcolmBaldrige

Applicant

MalcolmBaldrige

Applicant

BestPracticeExchange

BestPracticeExchange

InternalAssessment

InternalAssessment

InternalAssessment

InternalAssessment

CAPE*Applicant

CAPE*Applicant

IndustryWeek

Top 25Finalist

IndustryWeek

Top 25Finalist

IndustryWeek

Top 10Finalist

IndustryWeek

Top 10Finalist

1998 2002 2003 20042000 20011999

Incorporationof high-leverage

actions intostrategicplanning

TrainedExaminersdevelopedin-house

Deploymentof best

practicesto new

sitetenants

Quest forExcellence

andspeaking

engagements

ISOISO

InternalAssessment

InternalAssessment

Use CAPEfeedback to

improve usinginternal

Examiners

CCETeam

ExcellenceApplicant

CCETeam

ExcellenceApplicant

CMMICMMI

CCETeam

ExcellenceApplicant

CCETeam

ExcellenceApplicant

2005

Quality JourneyQuality Journey

IssueIssue

�� Requirements verification/validationRequirements verification/validationoccurring late in development life cycleoccurring late in development life cycle�� Higher costsHigher costs�� Schedule delaysSchedule delays

RequirementsRequirements DesignDesign ImplementationImplementation

System TestSystem Test Flight TestFlight Test

SolutionSolution

�� Utilize NonUtilize Non--advocate Review Team to:advocate Review Team to:�� Perform root causePerform root cause�� Identify areas for improvementIdentify areas for improvement�� Make recommendations based on diverseMake recommendations based on diverse

corporate knowledge from multiple programscorporate knowledge from multiple programs

DefinitionsDefinitions

�� Independent AssessmentIndependent Assessment -- An impartial and inAn impartial and in--depthdepthanalysis of a major issue or key milestone eventanalysis of a major issue or key milestone eventperformed by an Independent Assessment Review Teamperformed by an Independent Assessment Review Team

�� NonNon--Advocates: Subject Matter Experts (Advocates: Subject Matter Experts (SMEsSMEs) from any) from anyof the following groupsof the following groups�� Boeing nonBoeing non--program employeesprogram employees�� Outside consultantsOutside consultants�� Industry SMEIndustry SME�� FellowsFellows�� NonNon--program related customersprogram related customers�� Third party examinersThird party examiners

NAR ProcessNAR Process

�� Identify nonIdentify non--advocate teamadvocate team�� Define scope of reviewDefine scope of review�� Data collectionData collection�� Analyze dataAnalyze data�� Develop final report/Develop final report/outbriefoutbrief

CC--17 Application of NAR17 Application of NARProcessProcess

Identify NonIdentify Non--advocate Teamadvocate Team

�� Selected from local siteSelected from local site tenentstenents�� BB--1B, C1B, C--130 AMP, C130 AMP, C--1717

�� Utilized pool of Technical FellowsUtilized pool of Technical Fellows�� Boeing recognized technical experts in variousBoeing recognized technical experts in various

skills (i.e. Systems Engineering,skills (i.e. Systems Engineering,Communications)Communications)

�� Select chairpersonSelect chairperson�� BB--1B Chief Engineer1B Chief Engineer

Define Scope of ReviewDefine Scope of Review

�� Software Development processSoftware Development process�� Systems Engineering processSystems Engineering process�� Validation and Verification processValidation and Verification process�� Project ManagementProject Management

ExpectationsExpectations

�� Identified expectations of upperIdentified expectations of uppermanagementmanagement�� Process issuesProcess issues�� Improvement opportunitiesImprovement opportunities�� Lessons learnedLessons learned�� RecommendationsRecommendations

Resources RequiredResources Required

�� DataData�� Access to project personnelAccess to project personnel�� War RoomWar Room -- FacilitiesFacilities�� NAR Team availability and scheduleNAR Team availability and schedule

Data CollectionData Collection

�� DocumentationDocumentation�� DeliverableDeliverable�� NonNon--deliverable (Engineering Notes)deliverable (Engineering Notes)�� Software Development Folders (SDF)Software Development Folders (SDF)

�� InterviewsInterviews�� Engineers, Managers, Project managersEngineers, Managers, Project managers�� CustomersCustomers�� SuppliersSuppliers

Analyze DataAnalyze Data

�� Lack of process complianceLack of process compliance�� Lessons learnedLessons learned�� Process improvementProcess improvement�� Lean engineering opportunityLean engineering opportunity

Develop Final Report/Develop Final Report/OutbriefOutbrief

�� Summarize issuesSummarize issues�� Provide recommendationsProvide recommendations

�� NearNear--termterm�� LongLong--termterm