agile, cloud computing, open source and what's in between

175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir

Performance Testing in the

Agile Age


Alon Girmonsky About The Speaker

•  Hands-On Developer for the Past 20 Years •  Expertise

– Large-Scale Web and Mobile Applications –  IP & Internet

•  Founder & CEO of BlazeMeter – A Load Testing Platform as a Service – 100% Compatible with Apache JMeter

USERS

20,000

TESTS

500,000


The Agile Enterprise

•  All organizational divisions - not just R&D •  It’s about survival in a highly competitive

business environment •  Innovate in short Iterations – a MUST! •  New DNA •  New IT Infrastructure (a.k.a Devop)


Agility Enablers

Open Source

Cloud Compu/ng

E.G E.G


Performance Testing

Waterfall Agile

Front-‐end

Back-‐end

Func/onal

Load E.G API

E.G Java Script

In The Agile Age

2005 2010 2014


Agile? What it Means?

"We'd like to squeeze in a load test for XXXX’s biggest event of the year, Fantasy Football season. Our best opportunity to execute this is tonight 3-7am EST. We'd like to generate a constant 40,000 hits/sec for about 1 hour. .."

Traditionally this would have taken weeks and cost $$$

An Example from a Large Media Company


Agile? What it Means?

Traditionally this would have taken weeks and cost $$$

Network Bottleneck Identified


For Both Mobile and Web

Massive Global Simulation of Users, Devices and Mobile Networks

Agile Performance Testing


480 CPU Cores and

1.8 TB of Memory

What Beast Has .. ?


480 CPU cores and 1.8 TB of Memory A Typical Test

EC2 Large 2 cores, 7.5GB





EC2 XL 4 cores, 15GB

X 200

X 20


Mobile Application Simulation A Common Use-Case

Using Agile Performance Testing

•  You can create such test in a matter of minutes •  You can run dozens of times per day on demand •  You can enjoy an interactive reporting that provides an

immediate intelligence

Actions Sign-up Sign-in Purchase Browse


Comprehensive scripting at your fingertips

Any Script, Any Scenario

•  HTTP / HTTPS •  Login / Signup •  Form POSTing / Credit card transactions •  Web services •  Parameters extraction •  Mobile •  Mongo, MSSQL, Oracle SQL, MySQL •  Multiple user profiles

» Any script » Any scenario » Any protocol


Agile Summary

1 to 300,000 Concurrency

Open-Source Compatible

Self-Served

TTS < 10m

Any Protocol

Any Scenario

Multi Locations

On-Demand

Comprehensive


The Load Sensitivity Point What to look for?


Load Performance Types

Perfect Time

Sensitive

Bad



Perfect

The Perfect Load

Time



Sensitive

Sensitive to Load

Time



Bad

Bad Performance

Time



Perfect

Here They Are Again

Time

Sensitive

Bad

175 Varick Street NY, NY 10013|1.855.445.2285|[email protected]

What Can Be Achieved By Using Agile Performance

Testing

Apache JMeter + AWS or Commercial Vendors

Use Case Examples


What to Look Out For? GEO Sensitive Testing

•  Realistic Vs Unrealistic Simulation •  Single Location Vs Geo Distributed Load •  Small Subset of Target Servers •  Unrealistic Connectivity


Testing AKAMAI CDN GEO Sensitive Testing


Testing AKAMAI CDN (ALL) GEO Sensitive Testing


Testing AKAMAI CDN (VA Only) GEO Sensitive Testing


In Real-time Pinpoint Accuracy


In Real Time Pinpoint Accuracy

50%-50%

60%-40%


In Real Time Pinpoint Accuracy

100 hits/s

1,000 hits/s


Benefits Distributed Load

•  More realistic •  More load •  Evenly distributed •  More IPs (For a more even load balancing) •  More locations (For GEO sensitive systems)


Conclusions Agile Performance Testing

•  Cloud Computing + Open Source = Agile •  Any Protocol, Any Scenario, Any Scale •  Self-Served, On-Demand, TTS < 10m •  For Dev, Devop, Performance Engineer and Ops •  Test Driven Development •  Better Quality + Faster = Competitive !


Thank you for your time!

Jolt Awards 2013: The Best Testing Tools

agile, cloud computing, open source and what's in between

Technology