testing in the cloud
DESCRIPTION
TRANSCRIPT
Testing in the Cloud
Our experience of testing software that is “out there”
Introduction
- The Presenters- Paul Selway – [email protected] Scott Rostal – [email protected]
- What is “The Cloud”?- What is different / the same for cloud testers?- Testing Philosophy / Test phases- What was easier / harder than expected?- Key success factors- Questions?
What is “The Cloud”?
• Internet (“The Cloud”) accessible solution
• Off-premise multi-tenant architecture
• Accessed via a browser (PC or Wireless)
• Delivered as Software as a Service (SaaS)
Common cloud companies are:
Google, Amazon, WorkDay, and Salesforce
What things are the same about testing cloud solutions?
Requirements coverage based testing Risk-based structured test approach Test Plan / Test Phases Test Cases / Test Data / Test Automation Defect Management / Functional Test Use of off-shore resources / Virtualization Politics {Quality / Schedule / Resources}
What is different about testing cloud solutions?
Shared multi-tenant test environmentsSecurity (Test User IDs / SSO)Integration of on/off premise systemsPerformance / volume testDefect isolationDocumentation of “Out Of the Box” reqsRelease Management
Project Philosophy
• Risk-based structured testing approach {std}
• “Exploratory Testing” to discover the out of the box cloud solution features {non-std}
• Data Migration – Moving legacy data off premise {Non-Std}
• Performance / volume test only our new infrastructure – not the cloud app {Non-Std}
Testing Phases
Overlapping Test Phases
Unit / CodeCoverage
Inspections& Reviews
Exploratory& System
UAT &Pilot
End-To-EndIntegration
A mix of waterfall (with heavy phase overlapping), and iterative delivery
Load &Performance
DataMigration
What was harder than expected?• Isolating the architectural components for a
performance test• Understanding that a cloud solution vendor
delivers new releases which you have to take!
• Accepting that you have a “black box” in your environment
• Determining who owns defects • Data Migration Mock Runs
System Integration Architecture
Third Party
Home Office
Front OfficeFinancial Planning
4
HO Notify
2
5
3
Field Portal
Generic Mainframe Processing
324
Downloads / Updates
DB7
TriggerSynched
Download Requestor
Request Family
Request by CIF
ID
Request Weekly
Calendar
Member Download
Cert. Value Download
Request In-Force Illustration
10
Computer Application
AM Tool
Outbound
notifications
Webmethods
Web
Ser
vices
API
Customer Data
Customer ReusableServices
Web ServicesData
Warehouse
2
Field platform Integration
6
Staging Server
Informatica
Updates from HO to AM Tool(Customers, Assignments, Contracts, Users, Agents, Alerts
Updates from AM Tool to HO(Activities for WAT)
Informatica Adapter for Salesforce
1
10
Mobile Server
11
Security
Security
Gateway10
5
1
7
4
9
54
3
9
7
8
21
6
8
8
9
6
3
Integrated Apps
1
Not Tested due to Sunsetting
Not Tested due to Existing Infrastructure
Manual Testing
Implicitly Performance Tested
Performance Testing
What was easier than expected?
• Getting the IT leaders to embrace change
• Overlapping Test phases
• End User Adoption
• Use of off-shored test resources– Functional– Performance– Data ETL
Key Success Factors
• Learn the “Out of the Box” cloud application features, security model, and configuration capabilities
• Understand the holistic system architecture
• Understand the migrated test data
• Establish tight communications with the vendor
Questions?
Contact Info
• Scott Rostal – [email protected] Lead, Thrivent Financial for Lutherans
• Paul Selway – [email protected], Red Path Consulting Group