agile testing framework - the art of automated testing
DESCRIPTION
Once your organization has successfully implemented Agile methodologies, there are two major areas that will require improvements: Continuous Integration and Automated Testing. This presentation illustrates why it's important to invest in an Automated Testing Framework (ATF) to reduce technical debt, increase quality and accelerate time to market. Learn more at www.agiletestingframework.com.TRANSCRIPT
![Page 2: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/2.jpg)
Agile Coach● Finance: American Express, Charles Schwab, Morgan Stanley,
Bank of America● Healthcare: Mayo Clinic, Phoenix Children's Hospital, Wolters
Kluwer ● Hospitality: Choice Hotels International, SkyTouch Technology● Software: JDA Software Group, Apriva Mobile● Security: LifeLock● Energy: First Solar● State of Arizona - First Things First, ADOT, ADE● Insurance: AAA Insurance
Facilitator of “The 7 Habits of Highly Effective People”
Certifications● Project Management Professional (PMP)● Agile Certified Practitioner (PMI-ACP)● Certified ScrumMaster (CSM) ● Certified Scrum Practitioner (CSP)
www.AgileTestingFramework.com
IT Professional● CIO - Concord Servicing Corporation Software● Vice President Communications - PMI Phoenix● Director of Web Technologies - I-ology
About Dimitri Ponomareff
www.linkedin.com/in/dimka5
![Page 3: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/3.jpg)
Agenda
● Agile Methodologies● Agile Testing Quadrant● Quality● Technical Debt● The Testing Pyramid● Driven Development
○ FDD○ TDD○ BDD
● Agile Testing Framework
www.AgileTestingFramework.com
![Page 4: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/4.jpg)
Agile Methodologies
www.AgileTestingFramework.com
![Page 5: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/5.jpg)
The Triple Constraint
WaterfallPredictive Process
The plan creates cost and schedule estimates
Constraints
Estimates
Scope(requirements)
Cost Time
PlanDriven
AgileAdaptive Process
The vision creates feature estimates
Cost Time
Scope(features)
ValueDriven
When it doesn’t fit in Agile, we cut Scope, not Quality!
www.AgileTestingFramework.com
![Page 6: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/6.jpg)
5 Levels of Agile Planning
www.AgileTestingFramework.com
![Page 7: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/7.jpg)
Agile Testing Touch Points
● Release Planning○ Document dependencies○ Write - End to end flow tests○ Write - Workflow tests○ Write - Integration tests○
● Release Testing○ Run - End to end flow tests○ Run - Workflow tests○ Run - Integration tests○ Run - Performance and security tests
● Iteration Planning○ Review - Acceptance criteria
● Iteration Testing○ Write/Run - Acceptance tests○ Write/Run - Unit tests○ Write/Run - Component tests
www.AgileTestingFramework.com
![Page 8: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/8.jpg)
Agile Testing Quadrant
Are we building the right product?
Are we building the product right?
Business Facing
Technology Facing
www.AgileTestingFramework.com
![Page 9: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/9.jpg)
Agile Testing Quadrant
Functional, Story & Prototype Testing
Unit & Component Testing
Business Facing
Technology Facing
Exploratory & Usability Testing
Performance, Load & Security Testing
Critique Product
Supp
ortin
g th
e Te
am
www.AgileTestingFramework.com
![Page 10: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/10.jpg)
Agile Quality
QA is no longer the Sole Quality Gatekeeper!
● QA may play new role● Agile Quality requires new skills● Focuses on Prevention instead of Detection● Testing role may be done by others● Testing done upfront● Automation plays a bigger role● Focuses on xUnit testing
www.AgileTestingFramework.com
![Page 11: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/11.jpg)
Up front testing leads to lower costs and better quality
www.AgileTestingFramework.com
![Page 12: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/12.jpg)
Test FirstTest First
Quality focus on Test first vs. Test last
Test First
Continuous Integration
Refactoring
Functional Testing
Performance Testing
www.AgileTestingFramework.com
![Page 13: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/13.jpg)
The whole team is responsible for quality
Business● Product Owners / Product Manager● Subject Matter Experts
Technology● Architects ● Database Administrators● User Experience Designers● Operations/Support team members
Team● Developers● Testers● Business Analysts
www.AgileTestingFramework.com
![Page 14: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/14.jpg)
Is incurred by not developing in the correct way...
Technical Debt
Like Financial Debt - is dangerous if the incurred interest and the debt itself are not payed...
www.AgileTestingFramework.com
![Page 15: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/15.jpg)
Two ways of doing things!
Technical Debt
Clean and smart way - takes longer to implement but makes change easier in the future
Quick and dirty way - get your features sooner, but make the future changes very hard
www.AgileTestingFramework.com
![Page 16: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/16.jpg)
Technical Debt
http://blog.techdebt.org/interviews/156/interview-with-philippe-kruchten-on-technical-debt-rup-ubc-decision-process-architecture
“Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt.”Ward Cunningham (1992-03-26). "The WyCash Portfolio Management System".
www.AgileTestingFramework.com
![Page 17: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/17.jpg)
● Loss of Productivity● Increase in Testing● Postponed Releases● Code Duplication● Low Code Coverage● Increase in Bugs● Unreadable Code● Decreased Velocity● Using Old Libraries● Heavy Stress on Approaching Deadlines● Being Scared of Changing Anything● Evil Hacks Wrong Design● Wrong Choice of Technology
Symptoms of Technical Debt
www.AgileTestingFramework.com
![Page 18: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/18.jpg)
Continuous inspection with SonarQube Managing 7 axes of code quality or preventing the Developers' 7 Deadly Sins1. Potential Bugs 2. No Coding Standards 3. Duplications 4. Lack of Unit Tests 5. Bad Distribution of Complexity 6. Spaghetti Design 7. Not Enough or Too Many Comments
Strategic Design1. Strategic debt intentionally accumulated in a project.2. Conscious, proactive decisions with larger short term benefits.3. Focuses on architectural and/or business trade-offs.4. Forgo’s extensive architecture for increased speed to market or reduced overhead
Agile Engineering PracticesPair Programming, TDD, Continuous Integration, Automated Unit Tests, Automated Functional Tests, Automated Other Tests(Regression), Refactoring, Clear Definition of Done
Fixing the Technical Debt
www.AgileTestingFramework.com
![Page 19: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/19.jpg)
The Testing Pyramid
Source: http://www.slideshare.net/nashjain/inverting-the-testing-pyramid
Traditional Agile
www.AgileTestingFramework.com
![Page 20: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/20.jpg)
The DDs...
FDDFEATURE Driven Development
TDDTEST Driven Development
BDDBEHAVIOR Driven Development
www.AgileTestingFramework.com
![Page 21: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/21.jpg)
FDD - Feature Driven Development
● Is a client-centric, architecture-centric, and pragmatic software process
● Feature Breakdown Structure (FBS) instead of WBS● A feature is a small, client-valued function expressed in the form:
<action> the <result> by/for/of/to a <object>
www.AgileTestingFramework.com
![Page 22: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/22.jpg)
TDD - Test Driven DevelopmentIs a rapid cycle of testing, coding, and refactoring
● Relies on a very short development cycle
● Developer writes automated test case first
● Test case defines desired improvement or new function
● Next develops minimum amount of code to pass test
● Lastly refactors the new code to acceptable standards
Code refactoring is a "disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior"
www.AgileTestingFramework.com
![Page 23: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/23.jpg)
BDD - Behavior Driven Development
● Outside-in and pull-based - Wire-frame > Test Cases > Coding● Multiple-stakeholder, multiple-scale, high-automation ● Describes a cycle of interactions with well-defined outputs ● Results in the delivery of working, tested software that matters
www.AgileTestingFramework.com
![Page 24: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/24.jpg)
TDD, BDD and automated testing...
TDDBDD
TDD invests in automated Unit tests.BDD invests in automated Acceptance tests.
www.AgileTestingFramework.com
![Page 25: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/25.jpg)
BDD - User Story TraceabilityID: 27STORY NAME: Save a list of potential cars for later review
As a: BuyerI can: Add a car to my wish listSo that: I can review my top choices at a later time.
Acceptance Criteria
Scenario 27.1: Add a car to my wish listGiven: A potential buyer is logged inAnd: The car is available for saleWhen: The ‘Add to Favorites’ option appearsAnd: A buyer flags the car for the wish listThen: The car details are displayed in the wish list
Scenario 27.2: Review list of favorite carsGiven: A potential buyer is logged inGiven: A potential buyer has previously picked some favorite cars’And: A buyer clicks on “view my favorites’When: The buyer views the wish listAnd: The car is still available for saleThen: The buyer can view the car summary in the wish list
StoryAs a <role>, I can <activity>, so that <business value>
ScenariosGiven <context>When <event>Then <outcome>
www.AgileTestingFramework.com
![Page 26: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/26.jpg)
Agile Testing Framework (ATF)
www.AgileTestingFramework.com
![Page 27: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/27.jpg)
Agile Testing Framework
Agile Planning Stage
● BDD Tests written in Agile Project Management Tool
www.AgileTestingFramework.com
![Page 28: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/28.jpg)
Agile Testing Framework
Agile Implementation Stage
● BDD Tests Pulled/Run
● Data Driven/Functional Tests written
● Services Tests Run
● CI process run
● Quality, Build Status, Test Results Reported
www.AgileTestingFramework.com
![Page 29: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/29.jpg)
Agile Testing Framework
Agile Regression / Release Stage
● Regression Tests - Selenium/BDD
● Performance Testing - Front End/JVM/Load
● Security Testing
● Web Accessibility Testing
● Web Consistency Testing
www.AgileTestingFramework.com
![Page 30: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/30.jpg)
Agile Testing Framework
Agile Feedback Loop
● Technical Debt Management● Quality Metrics● Build Status● Code Coverage - Unit/Functional● Test Results● Code Reviews
www.AgileTestingFramework.com
![Page 32: Agile Testing Framework - The Art of Automated Testing](https://reader034.vdocuments.mx/reader034/viewer/2022042623/54c78c664a7959a4108b45a5/html5/thumbnails/32.jpg)
This presentation was inspired by the work of many people and we have done our very best to attribute all authors of texts and images, and recognize any copyrights. If you think that anything in this presentation should be changed, added or removed, please contact us.
http://creativecommons.org/licenses/by-nc-nd/3.0/
www.AgileTestingFramework.com