udvid din test portefølje med coded ui test og cloud load test
TRANSCRIPT
#CampusDays
Udvid din test portefølje med Coded UI test og Cloud Load testVS20011
Peter Lindberg - Consultant @ Alterate
@peterlindbergdk
www.linkedin.com/in/peterlindbergdk
www.alterate.dk
#C
am
pusD
ays
Topics
Test automation
Microsoft Test Manager and manual tests
Coded UI Test
Load Test in the cloud with Visual Studio Online
#C
am
pusD
ays
Why test?
We all make mistakes!
But also:
• Are we meeting the requirements and expectations?
• Does the hardware support the application?
• Does it work as expected?
The sooner the defects are discovered – the cheaper they are to fix
#C
am
pusD
ays
Why automate tests?
Cost reduction!
• Manual test is expensive
• Let manual testers spend the time on value adding tests• New features
• Exploratory testing
• “Time to market” – higher release cadence
• Safety as developer
#C
am
pusD
ays
Automated
UI Test
Integration
Test
Unit Test
Exploratory
testing
#C
am
pusD
ays
#C
am
pusD
ays
Microsoft Test Manager and manual tests
#C
am
pusD
ays
Visual Studio Application Lifecycle Management
#C
am
pusD
ays
Microsoft Test Manager (MTM)
Separate application
Fully integrated with Team Foundation Server• Test artifacts are Work Items in TFS
• Links to Product Backlog Items, Bugs etc.
• Queries
• Reporting
• Ties directly in to the developer world
#C
am
pusD
ays
Microsoft Test Manager (MTM)
A fully-instrumented, configurable test runner captures detailed records of steps performed, behaviors observed, and the status of each test step. Pause testing to report a bug, even if it’s not directly related to the current test. High-fidelity defect reports go directly to the development team, including all the technical data needed to reproduce and resolve the error. Record tests for later playback to speed test sessions and enable automation. (visualstudio.com)
#C
am
pusD
ays
What’s in it for the developers?
“captures detailed records of steps performed, behaviors observed, and the status of each test step”
“Pause testing to report a bug, even if it’s not directly related to the current test”
“High-fidelity defect reports go directly to the development team, including all the technical data needed to reproduce and resolve the error”
“Record tests for later playback to speed test sessions and enable automation”
#C
am
pusD
ays
MTM Work Item Structure
#C
am
pusD
ays
Demo
#C
am
pusD
ays
Coded UI tests
#C
am
pusD
ays
What are Coded UI Tests?
Automated tests interacting with the application through the user interface
For example used for testing
• Input validations
• Calculations
• “Shopping cart” functionality
• UI Elements
• Navigation
• ….
#C
am
pusD
ays
Supported configurations and platforms
Configurations
• Windows 7, Windows 8, Windows Server 2008 R2
• x86 and x64
• .NET 2.0, 3.0, 3.5, 4.0 and 4.5
Platforms
• Windows Phone Apps - (Only WinRT-XAML based Phone apps are supported)
• Windows Store Apps - (Only XAML-based Store apps are supported)
• Windows Forms + WPF
• IE 8, 9 and 10 - (9 and 10 only supported on the desktop)
• Chrome (recording not supported, only playback)
• Silverlight 5 (only supported with unsupported plugin)
• SharePoint, Dynamics web client – Fully supported
#C
am
pusD
ays
#C
am
pusD
ays
Test Automation
Test
Execution
Test
Script
#C
am
pusD
ays
Which tests to automate?
Is it often used functionality?
Does the UI change often?
Is the test code easy or difficult to maintain?
How often is the test expected to be run?
#C
am
pusD
ays
Coded UI test features
• Fully integrated into Visual Studio ALM
• Automatic code generation
• “Intelligent” Intent Aware Recording and Resilient Playback
• Execute on build server
• C# / VB.NET
#C
am
pusD
ays
Generate Coded UI Tests
• Automate steps from MTM action recordings
• Coded UI Test Builder
• Write them yourself in C# / VB.NET (advanced)
#C
am
pusD
ays
The generated code
TestMethod
#C
am
pusD
ays
UIMap
#C
am
pusD
ays
Coded UI Test Editor
#C
am
pusD
ays
Coded UI Test Editor
• Change names of test methods and UI elements
• Split steps in to new methods
• Move generated code to UIMap partial class for customizing
• Add pauses
• View and change properties
• Remove unwanted steps (duplicates etc.)
#C
am
pusD
ays
Demo
#C
am
pusD
ays
Load Test in the cloud with Visual Studio Online
#C
am
pusD
ays
#C
am
pusD
ays
What do we want to achieve with load tests?
• Will the planned platform support the expected load and peak loads?
• Will simultaneous users cause problems
• Server load (max. CPU / memory consumption etc.)
• When does it break and how?
Planning is key! Dialogue with Product Owner / stakeholder etc.
What are targets and objectives?
What are the success & acceptance criteria?
Concrete and measurable!!
There’s a big difference between 1 and 5 seconds when a web site load…
#C
am
pusD
ays
How?Generate
traffic to
application
Does it
perform as
expected?
Recreate the
performance
problems
Fix the
performance
problems
#C
am
pusD
ays
#C
am
pusD
ays
So when should I start planning and execute load teste?
Start early!
=
Save time
#C
am
pusD
ays
What can be included in load tests?
• Web test
• Unit tests
• Coded UI (possible – but not recommended)
Customize!
• Coded Web Performance test
#C
am
pusD
ays
#C
am
pusD
ays
To the Cloud!
• Load tests can be executed from the cloud with Visual Studio Online
• MSDN subscribers have 15,000 virtual user minutes included pr. month
• No need to set up a big test rig / infrastructure “on premises”
• Setup in Visual Studio – execute in Azure
• Visual Studio connected to Visual Studio Online
#C
am
pusD
ays
Visual Studio
Load test
web service
Result
database
Controller
Test agent pool
Agent Agent Agent
Agent Agent Agent
#C
am
pusD
ays
Demo
#C
am
pusD
ays
Q&A#Ask me about everything!
Join me at the Microsoft Booth the next 30 minutes @Meet The Experts
Dont forget to: Evaluate this session!
EVENT SPONSORER
EXPO SPONSORER
TRACK SPONSORER