performance testing secrets in context · load testing principles . project context is central to...
TRANSCRIPT
![Page 1: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/1.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
By:
Scott Barber Chief Technologist PerfTestPlus, Inc.
Created for:
Performance Testing: Throughout the Application Life-Cycle
![Page 2: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/2.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
Chief Technologist, PerfTestPlus, Inc. [email protected] www.perftestplus.com
@sbarber Co-Founder: Workshop On Performance and Reliability
www.performance-workshop.org
Scott Barber
Co-Author: Author: Contributing Author:
Books: www.perftestplus.com/pubs About me: about.me/scott.barber
Performance Testing: Throughout the Application Life-Cycle
![Page 3: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/3.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
This keynote is a sub-set of 7 days of instruction (about the same as a 1 semester university course)
I do not believe in “Best Practices” (I believe in problem solving via experience, experiment & education)
Everything in this keynote is based on personal experience in *some* context (but not yours)
Adapt concepts to your situation/context Do network with others who are here (I bet they have
great ideas too)
Follow–up with me (what works for you… or doesn’t)
I *like* questions!
Notes
![Page 4: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/4.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
![Page 5: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/5.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
System or application characteristics related to:
Speed: - responsiveness - user experience
Scalability: - capacity - load - volume
Stability - consistency - reliability - stress
What is Performance?
![Page 6: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/6.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
What mom tells people: I help people make websites go fast.
What is Performance Testing?
What I tell people: I help and/or teach individuals and organizations to optimize software systems by balancing: • Cost • Time to market • Capacity
while remaining focused on the quality of service to system users.
![Page 7: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/7.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
Performance vs. Load Testing?
![Page 8: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/8.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
![Page 9: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/9.jpg)
Who is Responsible?
![Page 10: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/10.jpg)
Where Does it Fit?
![Page 12: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/12.jpg)
To Prevent Poor Performance…
… don’t just react when it happens.
![Page 13: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/13.jpg)
An Ounce of Prevention…
![Page 15: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/15.jpg)
Conception to Headstone
Not
Cradle to Grave
The Performance Lifecycle is:
![Page 17: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/17.jpg)
… Do you like Swim Lanes?
![Page 18: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/18.jpg)
…Or maybe Circles?
![Page 19: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/19.jpg)
• Multi-User Responsiveness
• Resource Trending • Limit Identification
• Prepare for Prod • Tune/Optimize • Predict • Early Warning
Identification
• Single User Responsiveness
• Resource Management • Component-Level
Concurrency
• Current Capacity • Projections • Scalability Plan • Reliability DevOps &
Architects
Architects Analysts &
Scrum
Scrum & Load Team
Load Team & DevOps
Dev
Test Stage
Prod
…Or Annotated Pie Charts?
![Page 20: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/20.jpg)
…Relativistic Comparisons?
![Page 21: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/21.jpg)
…How about Colors?
![Page 25: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/25.jpg)
Application Delivery Activities
![Page 27: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/27.jpg)
![Page 29: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/29.jpg)
![Page 33: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/33.jpg)
A “Test-Driven” Application Performance Management Model
With:
![Page 35: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/35.jpg)
Units Stories Tiers Resources Goals
Assess Compare Investigate Accept Answer
Dev & Prod Times Resources Sizes Frequencies Dashboard!
Proactive Micro & Macro Establish Goals Update Targets
The four T’s stand for…
![Page 37: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/37.jpg)
![Page 39: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/39.jpg)
![Page 41: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/41.jpg)
![Page 43: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/43.jpg)
![Page 45: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/45.jpg)
![Page 47: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/47.jpg)
![Page 48: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/48.jpg)
![Page 50: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/50.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
![Page 51: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/51.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
![Page 53: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/53.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
![Page 54: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/54.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
Load Testing Principles Project context is central to successful testing.
Business, project, system, & user success criteria.
Identify system usage, and key metrics; plan and design tests.
Install and prepare environment, tools, & resource monitors.
Script the tests as designed.
Run and monitor tests. Validate tests, test data, and results.
Analyze the data individually and as a cross-functional team.
Consolidate and share results, customized by audience.
"Lather, rinse, repeat" as necessary.
![Page 56: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/56.jpg)
![Page 58: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/58.jpg)
Unit-Level Testing Tools (‘cause folks always ask)
FireBenchmarks; Performance testing addin for NUnit JUnitPerf; a collection of JUnit test decorators for performance
Firefox Performance Tester's Pack
HTTPerf
![Page 60: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/60.jpg)
Questions?
![Page 61: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/61.jpg)
Preventing Poor Performance with a little work…
every day… from every one.
![Page 62: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/62.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
Contact Info
Scott Barber Chief Technologist
PerfTestPlus, Inc E-mail: [email protected]
Blog: scott-barber.blogspot.com
Web Site: www.PerfTestPlus.com
Twitter: @sbarber
![Page 63: Performance Testing Secrets in Context · Load Testing Principles . Project context is central to successful testing. Business, project, system, & user success criteria. Identify](https://reader034.vdocuments.mx/reader034/viewer/2022051806/5fffaf723861cf6eec3893aa/html5/thumbnails/63.jpg)
© 2012 PerfTestPlus, Inc. All rights reserved. Throughout the Life-Cycle
Did we learn anything?
Review & Questions