building a strategy for mobile test automation success
TRANSCRIPT
© Utopia Solutions
Building a Strategy for Mobile Test Automation Success
Lee Barnes, CTO Utopia Solutions
© Utopia Solutions
Agenda
• What is the end goal? • Why do we need a strategy? • What do we need to consider? • What should a strategy look like?
Test Automation Strategy Overview
© Utopia Solutions 3
WIFI / Carrier Networks
Ultimate Goal
Automation Framework
Test Cases
Backend System
Single set of test cases executing across a diverse set of real devices
© Utopia Solutions 4
Test Automation Success Criteria
• Issue detection and recovery • Accurate verification • Unattended execution
Reliable
• Minimum sensitivity to application and test case changes
• Test case maintenance separate from automation code maintenance
Maintainable
• Test coverage expanded efficiently • Automated test cases created by non-automation
resources Scalable
© Utopia Solutions
SECTION 2 Why Do We Need a Strategy?
© Utopia Solutions 6
Test Automation Challenges
• Defining appropriate scope • Integrating with SDLC • Addressing technical issues • Reliability, maintainability, scalability
General
• Platform diversity • Device diversity • Device Access • Lack of skilled resources • Rate of technology change • Rapid development cycles
Mobile Specific
© Utopia Solutions 7
Why Test Automation Fails
Organizational • Lack of support • Unrealistic expectations • Disjointed org structure • Missing skill sets
Process • Immature testing process • Segregated processes • Lack of test environment / data
control
Technical • Poor tool support for AUT • Under or over design automation
approach
© Utopia Solutions
SECTION 3 What Do We Need to Know First?
© Utopia Solutions 9
Are We Ready?
© Utopia Solutions 10
Mobile Test Automation Readiness Assessment
People
• Structure • Support • Goals and
Expectations • Resources and Skills
Process
• Test Maturity • Test Assets • Process Integration • Test Environment /
Data
Tech
• Mobile Platforms • Device Access • AUT Technology • AUT / Tool
Integration
Assessment Topics
© Utopia Solutions
Where Do We Get the Information?
Assessment Input
Stakeholders
Existing Assets
Target Applications
© Utopia Solutions
SECTION 5 What Does a Strategy Look Like?
© Utopia Solutions 13
Start With a Plan
Scope
OS/Device Selection
Test Case Selection
Non-Testing Tasks
Technical Challenges
Device Access
Platform/Device Diversity
Object Identification
Process Integration
Type of Dev Cycle
Test Suite Maintenance
Execution & Analysis
© Utopia Solutions 14
Mobile Test Strategy Topics
Mobile Test Automation
Strategy
Process & Organization
Environment
Technical Resources
Scope & Roadmap
© Utopia Solutions 15
Automation Strategy – Process and Organization
• Political and financial support • Realistic expectations • Centralized automation function
Key Organizational Components
• Test objectives and coverage goals • Integration with SDLC → proactive
maintenance
Key Process Components
© Utopia Solutions 16
Automation Strategy – Environment and Technical
• Dedicated environment for execution • Controlled test data • Controlled code migration
Key Environmental Components
• Robust (and timely) test tool support for mobile platforms
• Efficient device access
Key Technical Components
© Utopia Solutions 17
Automation Strategy – Resources
• Roles • Skills • Time
Key People Components
• AUT test environment(s) • Test suite development • Test suite execution
Key Hardware Components
• Test tool • Supporting utilities
Key Software Components
© Utopia Solutions 18
Automation Strategy – Scope
• Test data setup • AUT / environment preparation • Complex / tedious results verification
Test Support Tasks
• Applications built on common technology • Applications with frequent releases & friendly UI’s
Application Selection
• Frequently tested • High test case to UI ratio • Predictable results • Easy to automate
Test Case Selection
© Utopia Solutions 19
Automation Strategy – Roadmap
• Begin with required organizational and process changes • Overlap with base framework and minimal coverage • Enhance and add test coverage over small iterations
Create Implementation Timeline
Phase n Phase 1
Continuous Process & Organizational Improvements
Base Framework
Initial Coverage
AUT 1
Framework Enhancements
Expand Coverage
Initial Coverage
Phase 2
AUT 2
Expand Coverage
Expand Coverage
Initial Coverage
Phase 3
AUT 3
© Utopia Solutions 20
Questions…
…. and Answers!
Direct future questions to: Lee Barnes Founder and CTO Utopia Solutions, Inc. Email: [email protected] Twitter: @USI_LeeBarnes LinkedIn: linkedin.com/in/leebarnes Blog: http://utopiasolutions.com/blog/