process improvement with test assessment models

8/14/2019 Process Improvement With Test Assessment Models

1/23

Proc ess Imp rovement WithTest Assessment Mod elsMa rwa n Zeined d ineTesting Pra c tic e Ma na g erLionb rid ge Tec hnolog ies

March 31 April 2, 2009


2/23


3/23

Why Should I Consid er TestProc ess Assessments?

Produce Higher Quality SoftwareRaise Awareness of Testing

Discover Strengths and WeaknessesIncrease Test Productivity

Boost Test EfficiencyInstitute Continuous Test ImprovementManagement DirectivesContractual Fulfillment Specific Projects


4/23

Why Use a TestImp rovement Mod el?

Model-Based Test Process Improvement:

An improvement effort that uses a model to appraisethe quality of test processes, identify and prioritizeneeded changes, and guide its improvementactivities.

Global AcceptanceStructured Framework: Measurement and ScoringCertifications and AwardsIncreased Improvement Efficiency and PredictabilityGreater Industry Recognition: Customer Realization


5/23

Risks of Using TestImp rovement Mod els

All models are wrong. Some are useful. George Box, Statistician

How wrong can a model be before it is useless?There is no silver bulletModel Evolution and Maintenance


6/23

Test Imp rovement Mod elSelec tion Criteria

Should improve the test process

Must have a maturity structureShould have sufficient information

Must be adequately maintained


7/23


8/23

Comp a ring TMM/ TMMi to TPITMM TPI

Type Model Maturity Maturity

Year of development 1996 1997

Approach Conceptual Practical

Maturity Model Foundation Gelperin and Hetzel None

Number of Levels 5 13

Number of KPAs 13 20

Assessment Type Questionnaire ChecklistAssessment Elements Assessment Procedure,

Questionnaire, Training andTeam Selection criteria

Assessment Guideline,Checklist and Test MaturityMatrix

Assessment Foundation CMM, ISO, SPICE Practical Experience

Information about Model Articles, Dissertations andBook

Articles, tools and Book

Sources: Burnstein & Koomen/Pol


9/23

Key Proc ess Area sTMM/ TMMi vs. TPITMM TPI

Test Planning Test StrategyTesting/Debugging Policies/Goals Life-cycle ModelBasic Testing Techniques/Methods Moment of InvolvementControlling and Monitoring Tests Estimating and PlanningIntegration of Test Test Specification Techniques

Test Training Static Test TechniquesTest Organization MetricsSoftware Quality Evaluation Test Automation (tools)Test Measurement Test EnvironmentReview Program Office EnvironmentTest Process Optimization Commitment and MotivationQuality Control Test Functions and TrainingDefect Prevention Scope of Methodology

CommunicationReportingDefect Management

Testware ManagementTest Process ManagementEvaluationLow Level Testing Sources: Burnstein & Koomen/Pol


10/23

Assessing TMM Ma turityLevels

Source: I. Burnstein, Practical Software Testing


11/23

Five Ma turity Levels of TMM

Source: I. Burnstein, Practical Software Testing


12/23

Three Pha ses of Test Proc essAssessment

Phase 1o Plan Assessmento Prepare Questionnaire

o Review Documentationo Schedule Interviews

Phase 2o Conduct Interviews

Phase 3o Deliver Assessment Findingso Provide Improvement Recommendation
http://www.movieweb.com/features/pop.php?p=PHseTvwxGGnQww


13/23

Assessment Planning

Define Assessment Scope/Goals

Assemble Assessment Team

Collect Collaterals to be Reviewed

Notify Stakeholders to beInterviewed

Define Expected Deliverables

Publish Assessment Questionnaire

Distribute Assessment Schedule

Milestones Sample Deliverables

Confirm assessment objectives, collect collaterals and notify stakeholders

Assessment Project PlanPresentation to Stakeholders

Notification Email toStakeholders

Assessment Questionnaire

Interview Schedule

1


14/23

AssessmentAna lysis & Interviews

Analyze test processes, practices & tools and conduct interviews

Review Received Collaterals

Review Questionnaire Answers

Solicit Missing or AlternativeCollaterals

Generate Additional Questions

Validate Presence vs. Awareness

Confirm Usage and Consistency


Updated Assessment ProjectPlan

Updated Interview Schedule

Progress Reports

Interview Matrix


15/23

Find ing s a nd Imp rovementRecommendations

Analyze additional collaterals and questionnaire answers and produce reports

Analyze Additional Collaterals

Review Interview QuestionnaireAnswers

Combine All Answers into ScoringSpreadsheet

Identify Strengths andWeaknesses

Provide Recommendations withComplexity and Priority Ratings


Assessment Findings ReportImprovement RecommendationDetailed Report

Improvement Roadmap

Findings Presentation toStakeholders

Professional Services Proposal


16/23

TMM Ma turity Results


17/23

TMM Level 2: Pha se DefinitionGoa l 1 : Develop Testing a nd Deb ugg ing Goa ls a nd Polic iesNot Sa tisfied


18/23

TMM Level 2: Pha se DefinitionGoa l 2 : Initiate a Test Planning Proc essSa tisfied


19/23

TMM Level 2: Pha se DefinitionGoa l 3 : Ba sic Testing Tec hniq ues a nd Method s

Sa tisfied


20/23

Imp rovement Roa d ma p


21/23

Imp lementa tion Benefits

Organization wide committee orgroup on test planning

Test planning procedures, goals,and policies are developed,documented, distributed, andmaintained.

Technical training and basic tools

are availableStaffing and on-boarding of test andQA staff is managed and governed

Consistent high quality test planningand execution collaterals


22/23

Imp rovement Benefits6-12 Months

Justifiable and Defendable StrategiesPrescriptive Guidelines and PoliciesConsistent Test Planning and ExecutionDefined Product Quality AttributesQuality Audit Guidelines

Monitoring and Control Policies.Fully Integrated Testing Activities into SDLCEstablished Training Protocols.

Consistent Quality Evaluation Metrics.Achieve TMM Maturity Level 3


23/23

Tha nk you!

For m ore inform a tion:

Ma rwa n Zeined d inePra c tic e Ma na g er, Ma na ged Test Serives

Lionb rid ge Tec hnolog ies(704) 528-3730

Ma rwa n.zeined d ine@lionb rid ge.c omwww.lionbridge.com

process improvement with test assessment models

Documents