performance: see the whole picture

1

Performance: See the Whole Picture

Alexander Podelko alex.podelko@oracle.com

alexanderpodelko.com/blog@apodelko

WebPerfDays New York 2013

October 17, 2013

2

About MeI have specialized in performance for the last

16 years

Currently performance testing and optimization of Hyperion products at Oracle

Board director at CMG http://cmg.org, organization of performance and capacity professionals– Conference November 4-8, 2013 in La Jolla, CA

Disclaimer: The views expressed here are my personal views only and do not necessarily represent those of my current or previous employers. All brands and trademarks mentioned are the property of their owners.

3

Agenda

Performance basics to make sure that we are on the same page

Are we taking care of all parts of performance puzzle?– Where do we place load testing in web

operations?

4

Performance

End-to-end, every factor impacts

Front-end– Network and client-side time

Back-end– Server time

Ignore subtleties for the 5 min talk

5

Front-End TimeTakes 80-90% of response time for modern

web sites– So indeed deserves the attention it is getting

now

Is constant, doesn’t change with load– Single user may investigate– Again ignore subtleties for the 5 min talk

Doesn’t matter for scalability

6

Back-End Time

Server processing

Change with load– You need apply load to investigate

Closely correlated with scalability

7

Front- vs. Back-EndBack-end response times depend on load

Thanks to Andy Hawkes for the graph

8

Performance Risk Mitigation I

Single-user performance engineering – Profiling, WPO, single-user performance

Software Performance Engineering– Modeling, Performance Patterns

9

Performance Risk Mitigation II

Instrumentation / APM / Monitoring– Production system insights

[Auto] Scalable Architecture

Continuous Integration / Deployment– Ability to deploy and remove changes quickly

10

But all of them don’t replace load testing

Load testing complements them an several important ways

11

Load Testing: Terminology

Any testing applying multi-user synthetic load to the system

– Load testing– Performance testing– Stress testing– Scalability testing– Volume testing– Reliability testing

– Concurrency testing– Endurance testing– Longevity testing– Soak testing– Stability testing

12

The StereotypeLoad / Performance Testing is:

– Last moment before deployment– Protocol Level Record-and-Playback– Large corporations– Expensive tools requiring special

skills– Lab environment– Scale-down environment – …

No ! It is just one case from numerous options !

13

Load Testing

Any way to apply multi-user synthetic load:

– Recording / playback on the protocol level

– Recording / playback on the GUI level

– API calls / requests scripts– System and components levels

14

Web Operations and Load Testing

“Our experiences were universal: Our software crashed or couldn’t scale”– Jesse Robbins, Foreword

Found 1 mention:– “In our opinion, the only way to make

sure our production systems could handle the load was to test them, live, during the day when traffic was flowing” Brian Moon, p.134

15

What Load Testing Adds

Verification that the system handles the load

Verification of multi-user performance

Performance optimization– Exactly the same load

Debugging/verification of multi-user issues

16

Can You Handle Peak Load?You can’t know without testing:

17

Single-User Performance Improvement

May lead to multi-user performance degradation

18

Missing Parts

Some important performance engineering areas are not covered much at Velocity– Modeling, load testing, etc.

Come to the Performance and Capacity conference by CMG http://www.cmg.org– November 4-8, 2013 in La Jolla, CA– Perfect complement to Velocity / WebPerfDays

19

TakeawaysSee performance end-to-end, the whole picture

Time spent is not the only variable– Load is another, it may be different under load

Still no way to verify that the system will work under the load but load testing– Let’s talk during open space

20

Questions?Alexander Podelkoalex.podelko@oracle.com

alexanderpodelko.com/blog@apodelko