edwin van loon - what's in the cloud for testing - eurostar 2012
DESCRIPTION
EuroSTAR Software Testing Conference 2012 presentation on cloud based testingTRANSCRIPT
Edwin van Loon, Valid
What’s in the cloud for testing?
The sky is the limit
www.eurostarconferences.com
@esconfs #esconfs
Introduction Edwin van Loon
• ISEB Practitioner
• Lean Six sigma green belt
• Allmost 15 years of experience
within different testing roles
• Living in Belgium
• Working in the Netherlands
edwinvanloon
Edloon
Cloud
Millennium change
Agile manifesto
ISO SQuaRE (25010)
Web based development
Introduction of the Euro coin
Model based testing
Risk based testing
Regulatory compliance
Test Certification
Agenda
•The Cloud in general
•The Cloud and Testing
•The cloud and it’s possibilities
• Testing the Cloud
What’s in the cloud for testing
Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General
Applications
Runtime
Security
Databases
Servers
Network
Storage
Hardware
Virtualization
Cloud layers
Applications
Runtime
Security
Databases
Servers
Network
Storage
Hardware
Virtualization
Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General
Cloud layers
Applications
Runtime
Security
Databases
Servers
Network
Storage
Hardware
Virtualization
Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General
Cloud layers
History of the Cloud
1997 2000 2002 2004 2008 2010 2012 -2015
Early SaaS
adaptors
16.3% compound
annual growth rate
through
Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General
Announcement Cloud in 1997 “In the last 7 years, do you know how many times I have lost any personal
data? Zero.
Do you know how many times I have backed up my computer? Zero…
I have computers at Apple, at NeXT, at Pixar, and at home. I walk up at any
of them and log in as myself, it goes over the network, finds my home
directory on a server, and I got my stuff where ever I am…
Carrying around these non-connected computers or computers with tons of
data and state in them is Byzantine by comparison.”
Steve jobs at WWDC 1997 (Apples Worldwide Developers Conference) (http://www.youtube.com/watch?v=3LEXae1j6EY)
Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General
Cloud computing - Advantages
1. Time to market
2. Scalability
3. Reduction of IT-costs
Testing the Cloud Cloud Possibilities Cloud and Testing Cloud General
Cloud testing
• a form of software testing in which
web applications use cloud computing
environments to simulate real-world user
traffic.
• allows you to use simulated web traffic to ‘load
test’ a website or web application with a
flexible, scalable and affordable testing
environment.
Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing
Test tools:
- Functional and Performance testing
- Cross browser testing
- Test management
Runtime
Security
Databases
Servers
Network
Storage
Hardware
Virtualization
Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing
Cloud test tools
History of the Cloud
1997 2000 2002 2004 2008 2010 2012 -2015
Early SaaS
adaptors
16.3% compound
annual growth rate
through
Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing
Advantages cloud test tools
Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing
• Accessible at Any place, Any time, Any device
• Scalability
• No infrastructural constraints
• No geographical constraints
• Low time to market
• Temporary contract / low costs
• No installation
• Availability
• Usability
• Possibilites for endurance testing
• ………
Testing the Cloud Cloud Possibilities Cloud General Cloud and Testing
Demo
Changed quality expectations
Cloud General Cloud and Testing Testing the Cloud Cloud Possibilities
Do you know how many times I have lost any personal data?
Do you know how many times I have backed up my computer? Zero!
Users of cloud solutions expect
• Being able to use the solution at any time, any
place using any device
• Being able to run their processes
without failures
• No loose of data
Cloud General Cloud and Testing
Test tools
Test portal
Templates
Standards
KPI’s
Virtual testmachines
Test servers
Network types (latency)
Testing the Cloud Cloud Possibilities
Test layers Testing as a
Service
Portal
Tactical
Tactical
TaaS (or Quality as a Service)
Operational
Software under Test
Define
Goals
Define
KPI’s Improve
Managed: In Control
Operational
Test
tools Standards
Test
results Metrics Test
machines
Define
Approach
Measure
Improve (continuous)
Consolidate
Advice
Infrastructure Test
machines
Test employees / test activities
Processes Standards Tools
Templates
TaaS Service Level agreement
• Agreements on:
• Number of test cycles per period (including peak and off-peak)
• Test coverage (quality level) or effort
• Through put time (test execution, test preparation, etc.)
• Defect detection percentage (DDP)
• Test or QA activities in general
• Reports (operational and tactical)
• But also ……
• Quality level of documents used for testing and the system under
test
Cloud General Cloud and Testing Testing the Cloud Cloud Possibilities
Testing the cloud
• Security:
• Compliancy statement
• Hack testing
• Continue monitoring
• Test of cost models
• Be aware of possible fixed release dates
of SaaS solutions
• Juridical aspects of the cloud
• Availability of cloud solutions: • Test of vendors (Cloudsleuth)
• Two internet providers
• Two instances / multi-region
Cloud General Cloud and Testing Cloud Possibilities Testing the Cloud
Resume
• The cloud is a TaaS enabler
• The cloud will change the expectation of users
regarding quality
• Cloud services requires different test approach
• Cloud test tools allow test teams to:
• avoid high costs on investments
• execute test global tests
• The cloud allows the test community to cooperate
and connect
The cloud has a already a high impact on
Software Testing
…….. and the impact will become huge
Stay Ahead