dev302. best practice visual studio ultimate web performance tests load tests load test rigs visual...
TRANSCRIPT
Identify and Fix Performance Problemswith Visual Studio Ultimate
Benjamin DayBenjamin Day Consulting, Inc.www.benday.com | blog.benday.com
DEV302
Benjamin Day
Consultant, Coach, TrainerProfessional Scrum Development Trainer
http://scrum.org
Unit testing enthusiastMicrosoft MVP for Visual Studio ALMSilverlight, Windows Azure, C#,Team Foundation Serverhttp://[email protected]
Thanks.
To my content reviewersJerri Chiu, MicrosoftDennis Bass, Microsoft
The Agenda
Basics of Web Performance TestsLoad TestsLoad Test RigsPerformance Explorer
Existing application Identify performance problemsGo from bad performance to goodASP.NET & WCF
Disclaimer: I’m going to use the term “load testing” imprecisely…a lot.
Here we go.
The Purpose of Load Testing
Identify the capabilities of your applicationPossible goals
ExploreVerify Find the limitsCrush
The Obligatory “Eat Your Vegetables”
Load test throughout the development cycle.Catch problems earlyEstablish performance baselineWatch for performance trendsCheaper to fix earlyPlan for performance
Best Practice
Cheaper To Fix Early
Early Less Early Nearly Done Production0
10
20
30
40
50
60
Time
Co
st t
o f
ix
Dose Of Reality
You know you should work that way.You probably don’t.This talk is for you.You have problems.How do you figure out what’s going wrong?
Inspiration for this talk
Real customer engagementSpent years developing a web applicationReleased it Seriously grumpy customersHuge hardware~20 simultaneous usersFixed it
4000+ simultaneous usersReduced hardware
The Tools
Visual Studio
Ultimate
• Web Performance Tests• Load Tests• Load Test Rigs
Visual Studio
Premium
• Profiling Tools• Performance Wizard
demo
Tour Of The Application
Disaster Strikes.
The application is deployed to production.Users are losing their minds.
Intermittent errors on the site.Site is slow.
Your boss is seriously angry.“Fix it!”
The Problem.
“No Repro”“Works on my box.”You don’t know what the errors are.The site seems to work fine.
Wild guess: it dies under load.You need to create some load.
The Plan.
Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code
Web Performance Tests.
The Plan.
Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code
What is a Web Performance Test?
Test type in Visual Studio 2010 UltimateSimulate a user using an ASP.NET application
HTTP traffic
Building block of Load Tests
Tip: Web Tests & User Stories
Web Tests should simulate a User StoryExample:
As an administrator, I need to search for a person by username so that I can deactivate that person’s account.As a customer, I want to order a pizza and pay with a credit card.
Helps to organize your suite
demo
Create a basic Web Test
Data-driven Web Tests
Run web test once per row in data sourceSimulate multiple users doing *similar* actionsData source types
OLE DB, CSV, XML
ParameterizationBetter tests, better coverage
demo
Web Tests in the Underperforming Application
Load Tests.
The Plan.
Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code
What are Load Tests?
Mix of Web Performance and Unit TestsSimulates a mix of users doing a mix of thingsSimulate
Network speedsDifferent browsersVarying user activity loads
Why would you put a Unit Test in a Load Test?
Helpful for testing WCFCumbersome to simulate WCF via Web Test
(You’d have to re-invent the wheel.)
Unit tests can re-use the WCF libraries(Done and done.)
Important for Silverlight and SOAhttp://wcfloadtest.codeplex.com/http://sqlloadtest.codeplex.com/
demo
Hey Kids, Let’s create some Load Tests!(Hooray!)
Running Load Tests.
The Plan.
Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code
Two ways to run load tests.
Generate load from Visual Studio 2010Uses only 1 core/processorLimited to 250 simulated users
Generate load from a Test RigFar better testSimulate lots more users
Load Test Agent Licensing
Microsoft Visual Studio Team System 2010 Test Load Virtual User PackSimulate up to 1000 virtual usersUsed to cost $$$
Visual Studio 2010 Load Test Feature Pack
Used to cost ~$3500.00 per 1000 simulated usersFree starting March 8, 2011Visual Studio 2010 Ultimate with MSDNSimulate unlimited amount of load
(Limited only by your hardware.)
http://www.microsoft.com/visualstudio/en-us/products/2010-editions/load-test-virtual-user-pack
Load Test Rig & The Application
Application Under Test
Generates Load
Coordinates & Gathers Information
Controller
Agent 1 Web Server
DatabaseAgent 2
Agent n
ControllerOrchestratesLicensingPerfMon Counters
Agent (1..n)Simulates users
The Plan.
Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code
demo
Run Load Tests on the Rig
Performance Profiling
The Plan.
Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code
Profiling Tools
Load Tests help you find issues in a system“Surface” issues
Profiling helps you find issues in the codeUnit Tests, Web Tests, Load Tests
RepeatableGood for targeted tuning
System under “natural” loadNot repeatableLooking for rough patterns
Profiling Methods• Gathers information at intervals (clock cycles)• No code modifications• Use this for first pass explorations
Sampling
• Modifies your code• Gathers detailed timing and count informationInstrumentation
• Multi-threaded code profilingConcurrency
• Object creation• Garbage collection.NET Memory
• Interaction between your application and SQL Server via ADO.NETTier Interaction
Load Test Performance Sessions Via Wizard
Must be only Web Performance TestsCan’t profile via Load Test RigASP.NET must be running in x86 mode
demo
Let’s go profile some stuff.
The Plan.
Script user activity with Web Performance TestsSimulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefullyFind code problems with the Performance WizardFix the code
Any last questions?
For More Information: Articles
http://tinyurl.com/3xjqgvz
http://tinyurl.com/2ulvvvr
For More Information: Microsoft Links
Visual Studio Performance Testing Quick Reference Guidehttp://vsptqrg.codeplex.com/ Content Index for Web Tests & Load Testshttp://tinyurl.com/ye97e52
For More Information: Microsoft Bloggers
VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/Ed Glas - http://blogs.msdn.com/edglas/Bill Barnett - http://blogs.msdn.com/billbar/Sean Lumley - http://blogs.msdn.com/slumley/Dennis Stone - http://blogs.msdn.com/densto/Mike Taute - http://blogs.msdn.com/mtaute/Neelesh Kamkolkar - http://blogs.msdn.com/b/nkamkolkar/
DEV Track Resources
http://www.microsoft.com/visualstudio http://www.microsoft.com/visualstudio/en-us/lightswitch http://www.microsoft.com/expression/http://blogs.msdn.com/b/somasegar/http://blogs.msdn.com/b/bharry/http://www.microsoft.com/sqlserver/en/us/default.aspxhttp://www.facebook.com/visualstudio
Resources
www.microsoft.com/teched
Sessions On-Demand & Community Microsoft Certification & Training Resources
Resources for IT Professionals Resources for Developers
www.microsoft.com/learning
http://microsoft.com/technet http://microsoft.com/msdn
Learning
http://northamerica.msteched.com
Connect. Share. Discuss.
Complete an evaluation on CommNet and enter to win!
Scan the Tag to evaluate this session now on myTech•Ed Mobile
Miscellaneous Tips.
Include Load Tests In Your Nightly Build
Create a Test List (*.vsmdi)Include the Test List in the buildCaptures trend data at 24 hours intervalsCompare the load test data with Excel
Static Code Analysis / FxCop
Help you to find issues in your codeShow you IDispose errorsRules are configurableUse your judgment about which are relevant for your team
Avoid Static Methods & CA1822
Static Code Analysis Rule CA1822 is evil.
Sacrifices maintainability for perfomance.Blog post: “Static Methods Are A Code Smell”http://bit.ly/cnEsuV
Only optimize known performance problems.
You’d be surprised by what *ISN’T* a performance problemDon’t spend a ton of time coding fancy solutions to performance problems you *THINK* you might haveFind and fix *REAL* problems
Tip: WCF Message Size
Data structure formatting can make a huge differenceExample: Banks and the States they operate inOption 1: Dictionary<string, string>
Key = Bank IdValue = State Abbreviation
Option 2: Dictionary<string, string[]>Key = State AbbreviationValue = Array of Bank Id’s
Option 2 was approximately 90% smaller
Tip: Add Performance Counters
Create Performance Counters (perfmon) in your appDo this early in the development cycleMore directed profiling of your app during Load TestsGood for managing your app when it’s in production
(Operations people love this.)
Using Performance Sessions on x64
You can’t profile x64 IIS processes from the IDE
Option #1: Profile from the command lineStart ASP.NET profiling using VSPerfAspNetCmd.exe Run the web test
Option #2: Run the ASP.NET process in x86 mode
How To Run ASP.NET in x86 Mode
Set “Enable 32-bit Applications” to true on your application’s AppPool
