ALL YOU NEED TO KNOW ABOUT

REGRESSION TESTINGDAVID TZEMACH

WWW.DTVISIONTECH.COMJAN 23 2016

OVERVIEWTHE PROBLEM WE HAVE AS SOFTWARE TESTERS, IS THAT THE LIFE CYCLE OF A SOFTWARE IS A VERY DYNAMIC. THEREFORE, EVERY PART IN CODE MAY CHANGE AND LEAD TO A NEW RISKS THAT ALREADY REMOVED DURING THE PREVIOUS TESTING CYCLES.THE SOLUTION FOR OVERCOMING THIS PROBLEM IS TO USE THE TESTING METHOD OF “REGRESSION TESTING” THAT PROVIDE A GREAT SOLUTION TO HANDLE THESE NEW RISKS

DEFINITION OF “REGRESSION” TESTINGTHE MOST SIMPLIFIED DEFINITION OF THIS TESTING METHOD, IS THAT WE USE IT TO “VALIDATE A FUNCTIONALITY THAT WAS ALREADY WORKING AND TESTED BY THE QA TEAM, STILL WORKS AFTER A NEW CODE THAT WAS ADDED OR AN OLD CODE THAT WAS MODIFIED”

WHEN TO USE THIS TESTING METHOD..? YOU SHOULD USE REGRESSION TESTS, WHENEVER YOU HAVE RISK THAT A SOFTWARE THAT ALREADY TESTED MAY BE AFFECTED BY A NEW CODE THAT WERE ADDED/CHANGED, EXAMPLES:

• A NEW FEATURE THAT WAS ADDED.• BUG FIXES.• CHANGES IN ORIGINAL REQUIREMENTS.• NEW REQUIREMENTS ADDED. • NEW PATCHES OR UPDATED.

THE FLOW OF REGRESSION TESTS

PHASE 1: ANALYZE THE CHANGEIN THIS PHASE YOU NEED TO ANALYZE HOW THE NEW CODE THAT WERE ADDED/MODIFIED IS AFFECTING THE APPLICATION, TO ACCOMPLISH THIS TASK, PLEASE ANSWER THIS BASIC QUESTION: • HOW THIS CHANGE EFFECT PREDEFINED TESTS FOR THE

ORIGINAL CODE?• IS THERE ANY INTEGRATED CODE THAT IS AFFECTED BY THE

CHANGE? • WHICH AREAS ARE MOST LIKELY TO BE AFFECTED BY THE

CHANGE? • HOW THIS CHANGE EFFECT THE SOFTWARE DOCUMENTATION?

PHASE 2: THE TESTING STRATEGY BASED ON THE RISK THAT YOU ANALYZED ON PHASE 1, YOU HAVE THE FULL INFORMATION THAT YOU NEED TO DETERMINE WHAT IS THE TESTING EFFORT THAT YOU NEED TO REMOVE THE RISK: TEST THE ENTIRE CODE - THIS STRATEGY IS THE BEST WHEN YOU HAVE THE TIME, MONEY AND HIGHLY TRAINED AUTOMATION TEAM THAT ALLOWING YOU TO RUN THE ENTIRE TEST PLAN INCLUDING THE OLD AND NEW TESTS. TESTING DONE IN HIGH RISK AREAS- THIS STRATEGY WILL BE USED WHEN THE TESTING TEAM IDENTIFIES THE AREAS THAT MOST LIKELY AFFECTED FROM THE CHANGE AND PROBABLY CONTAIN THE BIGGER RISKS.

PHASE 3: HOW TO RUN YOUR TESTS? THE TEST EXECUTION CAN BE ACCOMPLISHED IN 3 DIFFERENT WAYS: MANUAL TESTS – MANUAL TESTERS WILL RUN THE SELECTED TESTS MANUALLY, BUT FOR ME, IT’S THE LESS PREFERRED WAY TO HANDLE THIS TYPE OF TESTS. AUTOMATED TEST - THIS IS MY PREFERRED WAY, ALL TESTS WILL RUN AUTOMATICALLY AND WITHOUT A MAJOR INTERFERING OF THE MANUAL TESTERS. AUTOMATION AND MANUAL TESTING – COMBINATION OF THE TWO WORLDS, OLD TESTS WILL BE COVERED BY AUTOMATION, THE NEW TESTS WILL BE COVERED MANUALLY.

PHASE 4: REPORTING THE TESTING RESULTS PER REGRESSION CYCLE SHOULD BE REPORTED AND ANALYZED, ACCORDING TO FAILURES, WE CAN ADD/REMOVE/MODIFY TESTS TO INCREASE THE EFFICIENCY OF FUTURE TESTING CYCLE.

CONSIDERATIONS WHEN BUILDING

REGRESSION TESTS

WHAT IS THE TESTING COVERAGE..? WE ARE USING REGRESSION TESTS TO VALIDATE THAT NOTHING IS BROKEN IN THE APPLICATION WHEN CODE MODIFICATIONS ARE DONE, THIS FACTOR HAS A MAJOR EFFECT ON WHAT COVERAGE DO WE WANT TO HAVE, SHOULD WE ADD EVERY TESTS TO THE REGRESSION CYCLES OR NARROW IT TO A SPECIFIC SET OF TEST CASES…?

HOW MUCH EFFORT IS NEEDED ..? ASK YOURSELF THIS QUESTIONS:

1. WHAT IS THE AMOUNT OF TIME THAT IT WILL TAKE TO CREATE THE REGRESSION TESTS..?

2. WHAT IS THE AMOUNT OF TIME THAT IT WILL TAKE TO EXECUTE THE REGRESSION TESTS..?

3. WHAT ARE THE COSTS OF THIS EFFORT (BASED ON THE ANSWERS THAT YOU PROVIDE ON Q1/2)..?

MAINTENANCE CREATING THE REGRESSION TEST PLAN, IS PROBABLY THE EASY PART, THE HARD PART IS TO MAKE SURE THAT THIS TESTS WILL REMAIN UPDATED PERVERSION, THEREFORE, YOU NEED TO MAKE SURE THAT YOU UNDERSTAND THE IMPACT OF THE MAINTENANCE COSTS AND THE EFFORT THAT IT WILL TAKE FROM YOUR TEAM.

TEST PRIORITIZATIONREGRESSION TESTS MAY INCLUDE THOUSANDS OF TEST CASES, IN SOME SITUATIONS YOU SIMPLY DON’T HAVE THE TIME TO EXECUTE THEM ALL. THEREFORE, YOU MUST PRIORITIZE THE TESTS TO RUN IN THE ORDER THAT WILL GUARANTEE THAT YOU COVER THE MOST CRUCIAL TESTS FIRST.

CAN YOU AUTOMATE..?CAN YOU AUTOMATE YOUR TESTS OR NOT..? THE ANSWER TO THIS QUESTION WILL DETERMINE HOW MANY TESTS YOU CAN WRITE AND RUN PER TESTING CYCLE.

FOR ADDITIONAL KB’S PLEASE VISIT MY BLOG

WWW.DTVISIONTECH.COM