test automation: agile enablement for business ... · exploratory testing testing skills test data...
TRANSCRIPT
![Page 1: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/1.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Test Automation: Agile Enablement for Business
Intelligence Teams
Lynn WinterboerAgile Analytics Educator & Coach
@AgileLynn
www.WinterboerAgileAnalytics.com
![Page 2: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/2.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Lynn Winterboer
Colorado NativeGuest Ranch Childhood
Colorado College: Southwest StudiesUniversity of Denver: International MBA
Agile Analytics Trainer & CoachFocused on agile and data projects since the mid-90’sCBIP – Mastery Level, Certified Scrum Professional, SAFe-PMPO, CSM, CSPO, PMP
![Page 3: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/3.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Agenda
•Why is test automation important for agile data teams?
•Why aren’t all data teams automating their tests today?
•What is the path to test automation?
•What does simple DW test automation look like?
![Page 4: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/4.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Why is test automation important to agile data teams?
![Page 5: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/5.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com© Ken Collier, 2017, All Rights Reserved
Agile Demands Something DifferentAgile calls for small increments of “potentially shippable code”.
That means QA is essential on a regular, frequent basis.
How can we be sure that something we do
here
Doesn’t break something we did back
here
Test Suite Test Suite Test
Suite Test Suite
Test Suite
Test Suite Test
Suite
Manual testing quickly becomes infeasible.
The test suite grows larger each iteration.
![Page 6: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/6.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com© Ken Collier, 2017, All Rights Reserved
Testing is Central to Agile
Agile BI development is driven by tests
Acceptance criteria are the definition of “done”
Passing tests are the measure of “done”
Regression tests are the measure of “still done”
![Page 7: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/7.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com© Ken Collier, 2017, All Rights Reserved
Agile Teams Understand…
… that test automation is a key technical enabler
to “being agile”
![Page 8: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/8.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Why aren’t all data teams automating their tests
today?
![Page 9: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/9.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
DW/BI Test Automation Challenges
Data industry does not focus on testing as much as other disciplines:
Education
Team Members
Skills & Discipline
![Page 10: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/10.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
DW/BI Test Automation Challenges
Existing tools are not easily adapted to the data world:
Development Languages
Web, Mobile and UI Focus
![Page 11: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/11.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
DW/BI Test Automation ChallengesGood test data sets are hard to come by:
Large volumes (on small environments)
Shared test environments
Data sensitivity
Small sets need extra planning and design
![Page 12: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/12.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
What is the path to test automation?
![Page 13: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/13.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com© Ken Collier, 2017, All Rights Reserved
Exercise: Path to test automationIn small groups, discuss the following pre-requisites to test automation and decide on the first three steps a DW/BI team should take toward test automation:
Business Domain Knowledge Learning Culture Test Maintenance Repeatable Tests
Test Design Test Feedback Loops Dedicated Testers UI Tests
Exploratory Testing Testing Skills Test Data Sets Unit Tests
Test Environment Automated Builds Testing is a Team Sport Test Organization
Manual Tests Build Quality In Acceptance Tests Test Automation Tool
![Page 14: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/14.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com© Ken Collier, 2017, All Rights Reserved
Agile Testing Perspectives
High QualityDW/BI System
3) Tools &Automation
1) Attitudes &Culture
2) Skills &Practices
Testing is a Team SportLearning CultureBuild Quality In
Testing SkillsTest Design, Organization & MaintenanceRepeatable TestsFeedback Loops
Test EnvironmentBuild AutomationTest Data SetsTest Automation Tool
![Page 15: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/15.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com© Ken Collier, 2017, All Rights Reserved
Test Automation Pyramid
AppTests
Acceptance Tests
Unit & Component Tests
Manual Tests
Lisa Crispin & Janet Gregory, Agile Testing: A Practical Guide for Testers and Agile Teams
Push as many tests as possible to this layer
Write these in business domain language
Brittle tests that are affected by UI changes
Subjective user feedback is essential
![Page 16: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/16.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com© Ken Collier, 2017, All Rights Reserved
Good Places to Start§ Unit tests on new development
AGILE TEAMS WRITE TESTS BEFORE WRITING ANY CODE
§ Regression tests you wish you had, before building something new
§ Deployment tests to ensure each build migrated correctly
§ Painful test setup or test execution prone to error
![Page 17: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/17.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
What does simple data warehousing test automation
look like?
![Page 18: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/18.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com© Ken Collier, 2017, All Rights Reserved
Basic Approach
Component under test
SourceSchema
TargetSchema
ActualResults
TestData
Test Automation
Setup Execute Test Report
leave no trace
leave no trace
Clean Up
Static Test Data
ExpectedResults
Test Queries
load test data
compare results
executetests
based on specific input data
![Page 19: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/19.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
What Constitutes a Test?
Actual Result(from test)
Expected Result(queried or
pre-calculated)
![Page 20: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/20.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
What Does It Look Like?
![Page 21: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/21.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Testing FrameworkFunctionality and Components:
§ Example source system§ SQL Server and related model (“AdventureWorks”)
§ Screens for describing and creating test cases § Microsoft Access form
§ Test execution § SQL Server stored procedures (individual tests)§ Power Shell scripts (batch execution)
§ BI Reports and Dashboards for monitoring§ MS Power BI
![Page 22: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/22.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Demo Workflow1) Define a test (MS Access form) 2a. Execute a single test (SQL Server stored procedure)
3. Report on test results (MS Power BI)2b. Execute a batch of tests (PowerShell scripts)
![Page 23: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/23.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Define a Test (MS Access) – Expected Result
![Page 24: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/24.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Define a Test (MS Access) – Query Result
![Page 25: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/25.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Execute a Single Test (SQL Server)
![Page 26: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/26.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Execute a Batch of Tests (Power Shell)
![Page 27: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/27.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Report on Test Results (MS Power BI)
![Page 28: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/28.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Summary and Resources
![Page 29: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/29.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Session Intent
1) Why:Test Automation is key to DW/BI agile success
2) How:Automate a solid testing practice; don’t automate chaos
3) Show:DW/BI test automation is not that hard
![Page 30: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/30.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
DW/BI Test Automation ReferencesReference DescriptionAgile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousingby Dr. Ken Collier
Chapter 7: Test-Driven Data Warehouse Development
Agile Data Warehousing for the Enterprise: A Guide for Solution Architects and Project LeadersBy Ralph Hughes
Part V: Agile EDW Quality Management Planning
Automated Data Warehouse Testing: Beginner’s Step-by-Step Guideby G. Suden
Java-based test automation for simple data warehousing tests
Agile Testing: A Practical Guide for Testers and Agile Teamsby Lisa Crispin and Janet Gregory
Great introduction to agile testing for any type of development
More Agile Testing: Learning Journeys for the Whole Teamby Lisa Crispin and Janet Gregory
Includes several chapters on test automation as well as one dedicated to agile testing for DW/BI
![Page 31: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/31.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
DW/BI Test Automation ToolsTool Description
DBFitdbfit.github.io/dbfit/
Open source database testing tool
iCEDQicedq.com
QuerySurgeQuerySurge.com
ZuzenaZuzena.com
Test automation tools designed specifically for
Data Warehousing and related projects.
Informatica Data Validationwww.informatica.com/etl-testing
Accelerate and automate Informatica ETL testing
in both production environments and dev/ test
Analytix Data Servicesanalytixds.com
WhereScapewherescape.com
TimeXtendertimeXtender.com
DW automation tools that include test
automation capabilities.
Tricentis Toscawww.tricentis.com/tricentis-tosca-testsuite/
DW-friendly test automation tool
![Page 32: Test Automation: Agile Enablement for Business ... · Exploratory Testing Testing Skills Test Data Sets Unit Tests Test Environment AutomatedBuilds Testing is a Team Sport](https://reader030.vdocuments.mx/reader030/viewer/2022021801/5b1def4f7f8b9a397f8b82ab/html5/thumbnails/32.jpg)
© 2018 by Winterboer Agile Analytics � www.WinterboerAgileAnalytics.com
Questions?
Lynn WinterboerAgile Analytics Educator & Coach
www.WinterboerAgileAnalytics.comlynn@WinterboerAgileAnalytics.com
@AgileLynn
Special thanks for contributing to this presentation:Cher Fox, Brad Ewald, Ken Collier, Joe Bernardini, and Deborah Krinitzsky