canadian patrol frigate software maintenance testing by lcdr t.l. williams
TRANSCRIPT
![Page 1: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/1.jpg)
CANADIAN PATROL FRIGATESOFTWARE MAINTENANCE
TESTING
BY
LCDR T.L. WILLIAMS
![Page 2: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/2.jpg)
CPF SOFTWARE
• 1,100,000 SLOC
• 223 MODULES
• 10 YEAR DEVELOPMENT
• TESTED AS PART OF SIX YEAR AT SEA TRIALS PROGRAM
• BUILD 36 FINAL CPF CONTRACT BUILD
![Page 3: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/3.jpg)
CPF MAINTENANCE CONTRACT ISSUES
• GOVERNMENT OWNED CONTRACTOR OPERATED (GOCO)
• MOVE PGC TO HALIFAX
• CM REMAINED IN MONTREAL
• NAVY VERSION 1.0 PROMISE
• DND REORGANIZATION/STAFFING
• TRANSITION OF CPF SOFTWARE TO IN-SERVICE AUTHORITY
![Page 4: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/4.jpg)
COMMS
TECHNICALAUTHORITY
SOFTWAREMANAGER
SENSORS
PROGRAM GENERATION
CENTER
CM SUPPORT TEWA
ASWCOMMAND SUPPORT
TRACK MANAGEMENT
QAADMIN
ASSISTANT
SMaRT
NAVAL
COMMUNITY
HSSF ORGANIZATION
![Page 5: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/5.jpg)
TESTING
“TESTING SHALL BE AT THE SAME LEVEL AS THE CPF PRIME
CONTRACT”
CPF MAINTENANCE CONTRACT STATED:
![Page 6: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/6.jpg)
NAVY VERSION 1.0 PROBLEMS
• REQUIREMENTS
• WORK SPLIT BETWEEN SITES• POOR PLANNING• FACILITY SCHEDULING • VERSION RELEASE DOCUMENT
![Page 7: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/7.jpg)
NAVY VERSION 1.0 PROBLEMS (CONT)
• RELEASING AUTHORITY
• CONTRACTOR TESTING
• AT SEA TESTING
• NUMEROUS DEFECTS
• STAFFING/EXPERIENCE LEVELS
• NO MEANINGFUL METRICS COLLECTED
![Page 8: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/8.jpg)
SUCCESS OR FAILURE?
• POLITICALLY FACILITY WAS JUDGED TO BE A SUCCESS
• STANDING UP THE FACILITY– SUCCESS
• PRODUCING THE FIRST IN-SERVICE BUILD– FAILURE
![Page 9: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/9.jpg)
CHANGES TO FACILITY
• REQUIREMENTS GROUP STOOD-UP
• CONTINUOUS IMPROVEMENT GROUP ESTABLISH
• INDEPENDENT TESTING GROUP ESTABLISHED
• INCREASED STAFFING FOR SMaRT
![Page 10: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/10.jpg)
CHANGES TO FALICITY (CONTINUED)
• MINI-SYSTEM INSTALLED
• AUTOMATED CONFIGURATION MANAGEMENT
• CLOSER TIES TO REQUIREMENTS GROUP DEVELOPED
• TRANSITION OF SOFTWARE TO IN-SERVICE AUTHORITY
• FACILITY REORGANIZED
![Page 11: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/11.jpg)
TECHNICALAUTHORITY
CSST(H)
SOFTWAREMANAGER
ADMIN ASSISTANT
SOFTWARE SYSTEM
ENGINEER
SOFTWARE DEVELOPMENT
MANAGER
SPECIAL PROJECTS MANAGER
INTEGRATED SUPPORT
MANAGER
ASW/COMM/TM
TEWA/SENSOR
LINK
SUPPORT
COMMAND SUPPORT
QUALITY ASSURANCE
CONFIGURATION MANAGEMENT
INFORMATION TECHNOLOGY
TESTING
NAVAL COMMUNITY
HSSF(H) REORGANIZATION
![Page 12: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/12.jpg)
SOFTWARE SYSTEMS ENGINEER RESPONSIBILITIES
• ASSESS IMPACT OF SOFTWARE AND SYSTEM CHANGES
• COORDINATE SYSTEM STUDIES AND INVESTIGATIONS
• REVIEW TEST PLANS AND TEST PROCEDURES
• OBSERVE ALL FORMAL TESTS
![Page 13: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/13.jpg)
QUALITY ASSURANCE
• QUALITY MANAGEMENT– DOCUMENT, V & V SOFTWARE
PROCESS– INCREASE PROCESS CAPABILITY AND
MATURITY– RAISE QUALITY AWARENESS AND
PROVIDE TRAINING– CONTINUOUS IMPROVEMENT
![Page 14: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/14.jpg)
QUALITY ASSURANCE (CONTINUED)
• QUALITY CONTROL– VERIFY ALL DEFECTS AND
ENHANCEMENTS ARE DOCUMENTED AND TRACKED
– WITNESS SPR CLOSURES, DELTA, AND SYSTEM TESTS
– MONITOR SOFTWARE BUILDS AND DELIVERY PREPARATION
![Page 15: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/15.jpg)
CONTINUOUS IMPROVEMENT HIGHLIGHTS
• REQUIREMENTS REVIEW
• PEER REVIEWS
• TRAINING– 247 HOURS PROGRAMMERS– 173 HOURS TESTERS
• NEW VERSION RELEASE PROCESS
• INCREMENTAL BUILDS
• NEW TESTING PROCESS
![Page 16: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/16.jpg)
SOFTWARE TEST PROCESS
Beta Site Testing
SPR Closure
Program Debug
Shipboard At Sea Testing
Shipboard Alongside
Testing
Beta Site Testing
Development Testing
Operational Evaluation
System Test
Delta Test
Survivability Test
Version Testing
![Page 17: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/17.jpg)
BETA TESTING
• OCCURS ON A CONTINUOUS BASIS
• CFNOS AND CFNES INVOLVED
• DEFECTS ARE DOCUMENTED– ATTEMPT TO REPRODUCE ON PREVIOUS
VERSION
• FREE PLAY
![Page 18: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/18.jpg)
VERSION TESTING
• VERSION SURVIVABILITY TEST– COMPARES SYSTEM PERFORMANCE
WITH PREVIOUS VERSION PERFORMANCE
– FULLY AUTOMATED– FOCUSES ON MISSILE THREAT
PROFILES, TEWA PROCESSING– REPORT PRODUCED AND REVIEWED
![Page 19: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/19.jpg)
DELTA TEST
• INTEGRATION TESTING
• WHITE BOX FOCUS
• DEMONSTRATES ALL CHANGES FUNCTION CORRECTLY IN THE SAME BUILD
• TEST REPORT PRODUCED/REVIEWED
![Page 20: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/20.jpg)
SYSTEM TEST
• NORMALLY STATIC
• FOUR PHASES– EW/ASW (8 HRS)– AWW/RADAR (8 HRS)– FULL LOAD TRACKING (2 HRS)– OVERALL (6 HRS)
• REPORT PRODUCED/REVIEWED
![Page 21: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/21.jpg)
TESTING IMPROVEMENT S
• TEST CELL ENHANCEMENT STUDY– FULLY DOCUMENT PROCESS
• DEVELOMENT AND USE OF METRICS
• FORMALIZATION OF PEER REVIEWS
• AUTOMATED TESTING
• DEFECT REPORTING
• DEFECT DATA COLLECTING
• TESTING/PROGRAMMING TOOLS
![Page 22: CANADIAN PATROL FRIGATE SOFTWARE MAINTENANCE TESTING BY LCDR T.L. WILLIAMS](https://reader035.vdocuments.mx/reader035/viewer/2022062515/56649d0a5503460f949dc99c/html5/thumbnails/22.jpg)
CONCLUSION
• INITIAL TESTING POOR
• TRYING TO DO TOO MUCH AT ONCE
• TESTING PROCESS NOW IN PLACE
• OBTAINING REPEATABLE RESULTS
• FINE TUNING PROCESS STAGE