going cloudy? how to test saas? with kees blokland
DESCRIPTION
View webinar: http://www.eurostarconferences.com/community/member/webinar-archive/webinar-80-going-cloudy-how-to-test-saas The introduction of cloud computing has changed the playing field for testing. Testing needs to evolve and innovate to address the newly introduced risks that come with "going cloudy" with application services. How do we make sure that the continuity of services is guaranteed? In this webinar Kees Blokland introduces new solutions to tackle the new risks that arise with SaaS. How to use innovative combinations of testing techniques to cope with this phenomenon.TRANSCRIPT
Welcome to the 2013
www.eurostarconferences.com
This webinar is presented by Kees Blokland and he will discuss:
‘Going Cloudy? How to test SaaS?’
Join us on Twitter! @esconfs #esconfs
Going Cloudy?
How to test SaaS?
Kees Blokland
Polteq Test Services BV
The Netherlands
3
ISBN 978-1-937538-38-5
searching, recording, accounting, paying, writing,
reviewing, tracking, calculating, developing, listening,
analyzing, transmitting, learning, controlling,
purchasing, testing, alarming, changing, updating,
deleting, accessing, rejecting, correcting, studying,
booking, receiving, tracing, protecting, deciding,
managing, teaching, facilitating, identifying, copying,
removing, demonstrating, checking, showing,
selecting, subscribing, unsubscribing, sharing,
mailing, communicating, reading, playing, working,
meeting, gambling, shopping, storing, cross
checking, retrieving, configuring, sketching, saving,
accelerating, enhancing, creating, growing, checking
in, checking out, finding out, reaching, denying,
talking, designing, making, verifying, measuring
6
Surf
Transfer
Develop and Test
Operate and Manage Store
7
Surf
Transfer
Develop and Test
Operate and Manage Store
storage claim
80% unused
redundancy, proliferation limitations
environmentally unfriendly
8
Surf
Transfer
Develop and Test
Operate and Manage Store
standard software bandwidth
internet technology
virtualization
SOA
9
searching, recording, accounting, paying, writing,
reviewing, tracking, calculating, developing, listening,
analyzing, transmitting, learning, controlling,
purchasing, testing, alarming, changing, updating,
deleting, accessing, rejecting, correcting, studying,
booking, receiving, tracing, protecting, deciding,
managing, teaching, facilitating, identifying, copying,
removing, demonstrating, checking, showing,
selecting, subscribing, unsubscribing, sharing,
mailing, communicating, reading, playing, working,
meeting, gambling, shopping, storing, cross
checking, retrieving, configuring, sketching, saving,
accelerating, enhancing, creating, growing, checking
in, checking out, finding out, reaching, denying,
talking, designing, making, verifying, measuring
Surf
Transfer
Develop and Test
Operate and Manage Store
US: National Institute of Standards and Technologyhttp://www.nist.gov
Essential characteristics
On-demand service
Broad network access
Resource pooling
Rapid elasticity
Measured service
Deployment models
– private cloud
– community cloud
– public cloud
– hybrid cloud
Service Models
Software as a Service
Platform as a Service
Infrastructure as a Service
Continuity
Privacy
Multi platform
Legislation
Cyber crime
Impact organisation
Standards
143
Continuïty
Privacy
Multi platform
Legislation
Cyber crime
Impact organisation
StandardsPerformance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
14
Other customers
YOUR
Operational Profile
YOUR
Operational Profile
PLUS
Performance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
15
Everything over the web
The idea:
“it’s safe”
Home ground for
hackers
Performance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
16
Bring Your Own Device
No free choice of
device.
No control.
Performance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
17
Updates, patches, fixes, …
Planned and
controlled
Do I have a
choice?
Performance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
18
Where is my data?
And is that OK?
In house.
Somewhere…
Performance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
19
Performance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
20
Testing?
Check
Review
Trial
Interview
Proof of concept
21
Testing!
Check
Review
Trial
Interview
Proof of conceptTestenProefIntake
InterviewProof of concept
22
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
TestenProefIntake
InterviewProof of concept
23
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
24
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
25
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
26
Performance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
Architecture
From “individual” risks
to
“individual” test measures
27
Selection
Implementation
Production
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
28
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
29
Selection Criteria
Proof of Concept
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
30
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
31
Known measures
tuned and tweaked
New measures developed
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
32
Load Testing
YOUR
Operational Profile
YOUR
Operational Profile
PLUS
ACTUAL MOMENT
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
33
Elasticity
Load and stress.
Load and elasticity.
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
34
load
load test – ‘up’
extend?
200
charged100
charged
no
yes
path test
4
99
100
101
boundary values
‘up’
tc 1: use=99, pay 100
tc 2: use=100, pay 100
tc 3: use=101, pay 200
‘down’
tc1: use=101, pay 200
tc2: use=100, pay 100
tc3: use=99, pay 100
boundary values
load test – ‘down’
35
Any device – any platform
Multiplatform
testing.
Multiplatform
testing.
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
36
3997 distinct Android devices
http://opensignal.com/reports/fragmentation.php
37
Internet Explorer 6
Internet Explorer 7
Internet Explorer 8
Firefox 3.5
Firefox 3.6
Firefox 4
Safari 4
Safari 5
Chrome11
Opera11
Windows XP
Windows Vista
Windows 7
Windows 2003 Server
Windows 8
Windows CE
Linux
Unix
Mac OS Lion
Mac OS Snow Leopard
iOS
Android
Operating systems
Browsers
Multi-platform testing
Devices
Computer
Mobile phones
Tablet
PC
Macintosh
SUN
NOKIA …
Samsung …Windows Mobile
iPhone ...
…
MOTOROLA …
Blackberry …
ASUS ...
…
38
Any device – any platform
Multiplatform
testing.
Multiplatform
testing.
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
39
Legislation + Regulations
=
Test basis
Incidental testing.
Compliancy testing.
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
40
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selectionPerformance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
41
Continuous
End-to-End Test
Functionals
and
non-functionals
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
Continuity
Privacy
Multi platform
Legislation
Cyber crime
Impact organisation
Standards
Check
Intake
Trial
Interview
Proof of concept
43
Performance risks
Security risks
Availability and continuity
risks
Functionality risks
Maintainability risks
Legislation and regulations
risks
Suppliers and outsourcing
risks
Performance testing
Security testing
Testing for manageability
Availability and continuity
testing
Functional testing
Migration testing
Testing caused by
legislation and regulations
Testing in production
Testing during selection
Architecture
From “individual” risks
to
“individual” test measures
Testing starts earlier
Testing scope is widened
Testing will never stop
44
ISBN 978-1-937538-38-5
Thanks for listening!
@esconfs#esconfs
www.eurostarconferences.com
Go to www.eurostarconferences.com for more information