ahmed salijee developer advisor [email protected] dtl201
TRANSCRIPT
Improve Application Quality with Microsoft Visual Studio Team System 2010 Team Test
Ahmed SalijeeDeveloper Advisorhttp://dotnet.org.za/ahmeds [email protected]
DTL201
int Multiply(int a, int b){
return (a*b);}
The extensive reviews and tests carried out during the Ariane 5 Development Programme did not include adequate analysis and testing of the inertial reference system or of the complete flight control system, which could have detected the potential failure.http://sunnyday.mit.edu/accidents/Ariane5accidentreport.html
Can you relate
Tester
Developer
Generalist SpecialistManual Testing Some scripting
Creates scripts to set up lab, create data
Strong scripting skills
Some coding skills
Strong coding
Develops fully automated testing procedures
Expert coding skills
Tester Segmentation
Black Box Testing
White Box Testing
API Testing
VSTS 2010 Test Capabilities
Coded UI Test
Unit TestingWeb Test
Load Test
Test Runner
Test Case Management
Virtual Lab Management
Data Collectors (Historic Debugging, System Info.)
Team Foundation Server with Reporting
Generalist Specialist
FF Manual Test
Today..
Coded UI Test
Unit TestingWeb Test
Load Test
Test Runner
Test Case Management
Virtual Lab Management
Data Collectors (Historic Debugging, System Info.)
Team Foundation Server with Reporting**
Generalist Specialist
FF Manual Test
Iteration N
TEST
Sprint plan
Write tests for US1
Write tests for US2
DEV
Sprint plan
Build 1 Build 2 Build 3
Implement User Story ‘1’
(US1)
Test US1 & file bugs
Implement US2
Build 4
Fix bugs
Build 6 Build 7
Regress impacted
tests
Build 5
Implement US2
Fix bugs
Test US2 & file bugs
Verify fixes
Verify fixes
Sprint planning: Move user stories from Product Backlog to Iteration Backlog –
• User Story #1: “As a <user> I want to <do some thing> so that I can <achieve some goal>
• User Story #2: “As a <user> I want to…
Play Back Test Case
Create Test Plan
(config etc)
Run and Record Test
Case
Create Test Suite
Run Test Case
Create Test Case
Connect to TFS
Track
Test Case Management ScenarioGetting Started
Getting Started – Test Case Management, Running your tests
demo
TEST
Sprint plan
Write tests for US1
Write tests for US2
DEV
Sprint plan
Build 1 Build 2 Build 3
Implement User Story ‘1’
(US1)
Test US1 & file bugs
Build 4
Fix bugs
Build 6 Build 7
Regress impacted
tests
Build 5
Implement US2
Fix bugs
Test US2 & file bugs
Verify fixes
Verify fixes
Sprint plan
Write tests for US1
Sprint plan
Build 1 Build 2 Build 3
Implement User Story ‘1’
(US1)
Test US1 & file bugs
MTLM provides rich tools for planning test coverage of requirements.
MTLM helps testers understand what’s available with each build.• Requirements• Tasks• Bugs
Sprint plan
Write tests for US1
Key Value: Requirements Planning
Build 3
Implement User Story ‘1’
(US1)
Test US1 & file bugs
Test Plans and TFS items
ServerTCM Server
Plan “Iteration 2” Suite US1 Suite US2
Test Case ATest Case BTest Case CTest Case D
MTLM Client
Plan “Iter 1”
Plan “Iter 3”
TFS
User Story 2
Test Case A
Test Case B
Test Case C
Task X
Task Y
Task ZTest Case C
Shared Steps and Parametersdemo
Shared Steps
Parameters
Anatomy of a Test Case
Test case fields –• Title• Priority, Area, Iteration, …
Tabs for linked items and other info.
Steps tab includes –• Action for each step• Expected Results• Attachments
Parameterized DataA table of parameter values for iterated, data-driven testing.
Shared StepsA common sequence of steps shared by multiple test cases.
Developer Codes Tasks, Fixes Bugs
Checks In
Tester runs appropriate
tests
Build Kicked Off
Tester updates test settings to
collect data ***
Developer Reviews
WorkItems
Reproducible Bug Scenario
Tester Picks up New Build
Tester Verifies/
Closes Bugs
Tester Finds Bug
Tester Logs/Updates “Rich Bug”
Tester finds non
reproducible big
Tester Determines what test to
run**
Rich Bugsdemo
TEST
Sprint plan
Write tests for US1
Write tests for US2
DEV
Sprint plan
Build 1 Build 2 Build 3
Implement User Story ‘1’
(US1)
Test US1 & file bugs
Build 4
Fix bugs
Build 6 Build 7
Regress impacted
tests
Build 5
Implement US2
Fix bugs
Test US2 & file bugs
Verify fixes
Verify fixes
Test US1 & file bugs
Fix bugs
Verify fixes
Build 3
MTLM helps testers and devs tighten the bug loop.
Testers file rich, easy-to-repro bugs
Build 5
Testers quickly verify fixes by playing back a recording (ffwd)
Fix bugs
Test US1 & file bugs
Build 5
Verify fixes
Key Value: Virtuous Bug Cycle
TEST
Sprint plan
Write tests for US1
Write tests for US2
DEV
Sprint plan
Build 1 Build 2 Build 3
Implement User Story ‘1’
(US1)
Test US1 & file bugs
Build 4
Fix bugs
Build 6 Build 7
Regress impacted
tests
Build 5
Implement US2
Fix bugs
Test US2 & file bugs
Verify fixes
Verify fixes
Key Value: Catch Regressions
Build 3 Build 4 Build 6 Build 7Build 5
Test US1 Code churn???
MTLM helps testers identify tests impacted by code churn
??
Build 4 Build 6 Build 7Build 5
Regress impacted tests
?
Recap: What did we accomplish?Rich Bug
Turned on rich data collectorsManually ran a test case -recorded our actions for future playback
Filed a rich, easy-to-repro bugTook new build with the resolved bugQuickly verified the fix using playbackClosed the bug
Regress Impacted TestsAssigned a build to our test planRan some test and marked them ‘passed’Checked-in churned codeAssign the new build to our test planEvaluated and reset tests impacted by code churn
Managing Testing Recap
Author and manage tests through plans, suites, and test cases Define testing matrix using configurationsControl data collection via test settingsRun tests via manual testing or automationCreate rich bugs with nearly no overheadFast-forward through tests to verify
Setup Environment (Agents and Controllers)
Convert to Automated
Define Automated
Test Settings
Run as part of Build Process
Associate Test Settings and
Build with Test Plan
Manual Test
Associate Automation
with Test Case
(Test/Lab Mgr)
Moving to Automated
Run TestsTrack and Report
Other Tests (eg Unit Tests)
Other Tests (eg Unit Tests)
Automated Testingdemo
Automated Testing Recap
Specialist testing experienceCoded UI Test TypeGenerate tests using manual test recordings, direct recording or control mapCreate C# or VB.NET codeLeverages test framework for data binding, data collection, test categories, etc.Leverage Team and Test Agents
Reporting Work Progress
Product Quality
Build Quality
Dashboard: Test Progress
Platform Support
Fully supported platform
Partial solution, further work required to completeBest efforts, may work with known issues, no ongoing investmentCurrently no support but on the roadmap for future releasesCurrently no support and none planned, opportunity for partners
Platform Beta 1 RTM RTM+ Notes
IE7/8 – HTML/AJAX
FF3 – HTML/AJAX CTP at VSTS 2010 RTM
IE 6
Chrome/Opera/Safari
Silverlight 3.0
Flash/Java
Windows Forms 2.0+NetFx Controls fully supported, working on 3rd party support
WPF 3.5+NetFx Controls fully supported, working on 3rd party support
Windows Win32
MFC
SharePoint
Office Client Apps
Dynamics (Ax)
SAP
Citrix/Terminal Services
VSTS 2010 Team Test Recap
Rich bugs made easyTesters are first-class citizens of ALMDevelopers and Testers have tighter interactionVery easy to create robust automation and re-factor codeExtensible platform
Resources
http://dotnet.org.za/ahmedsWill post links, resources here
http://blogs.msdn.com/mathew_aniyan http://blogs.msdn.com/amit_chatterjeehttp://blogs.msdn.com/vstsqualitytoolshttp://msdn.microsoft.com/en-us/library/ms182409(VS.100).aspx
VSTS/SDLC Sessions at TechedTime Code Title
Monday 9:15 DTL203 What’s New in Team Foundation Server 2010?
Monday 10:45 DTL305 Managing Releases Between Your Development and QA with Team System 2008
Monday 13:15 DTL201 Improve Application Quality with Microsoft Visual Studio Team System 2010 Team Test
Monday 13:15 OFC309 Capacity Planning for SharePoint Server 2007 using Visual Studio 2008 Team Test
Monday 15:45 DPR201 The Daily Scrum
Monday 17:15 DTL301 Power Tools on Team Foundation Server 2008
Tues 8:00 DTL205 A Lap Around Team System 2010 Architecture Edition
Tues 9:15 DTL306 Team Build Tips and Tricks
Tues 12:00 DTL313 Using Virtualization to Improve Application Quality with Team System Lab Management
Tues 13:15 DTL210 Managing Requirements with Team Foundation Server 2010
Tues 14:30 WTB212 How Microsoft and Others Use Team Foundation Server (whiteboard)
Tues 16:15 DTL202 Team System 2010 Development Essentials
Tues 16:15 ARC203 Application Lifecycle Maturity
Wed 10:15 DTL303 Practical Web Testing
Complete a session evaluation and enter to win!
10 pairs of MP3 sunglasses to be won
www.microsoft.com/teched
Sessions On-Demand & Community
http://microsoft.com/technet
Resources for IT Professionals
http://microsoft.com/msdn
Resources for Developers
www.microsoft.com/learningMicrosoft Certification and Training Resources
www.microsoft.com/learning
Microsoft Certification & Training Resources
Resources
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.