building robust and reliable software jason anderson, microsoft [email protected]

Building Robust and Reliable SoftwareJason Anderson, Microsoft

[email protected]

Agenda

• Team System Overview• Static Analysis• Profiling• Unit Testing• Test Driven Development• Code Coverage• Team Developer and Team Foundation

Server• Questions

Development Teams

InfrastructureInfrastructureArchitectArchitect

SolutionSolutionArchitectArchitect

Project ManagerProject Manager

DeveloperDeveloper

TesterTester

End UserEnd User

Visual Studio Team SystemPro

cess

an

d A

rch

itect

ure

Pro

cess

an

d A

rch

itect

ure

G

uid

an

ceG

uid

an

ce

Vis

ual S

tud

io In

du

stry

V

isu

al S

tud

io In

du

stry

Part

ners

Part

ners

Dynamic Code Analyzer

Visual Studio

Team Architect

Static Code Analyzer

Code Profiler

Unit Testing

Code Coverage

Visio and UML Modeling

Team Foundation Client (includes CAL)

Visual Studio Professional Edition

Class Designer

Load Testing

Manual Testing

Test Case Management

Application Designer

Logical Infra. Designer

Deployment Designer

Visual Studio

Team DeveloperVisual Studio

Team TestDynamic Code Analyzer


Code Profiler

Unit Testing

Code Coverage

Change Management

Work Item Tracking

Reporting

Project Site

Visual Studio

Team Foundation Integration Services

Project ManagementBig Build

Static AnalysisStatic Analysis

Static Analysis

• Find errors in your code before you run or deploy it

• Checks range from style to code correctness to security issues

• Integrated into the Team System Build Environment

Static AnalysisThe dev process without Static Analysis

CompileCompileCompileCompileCodeCode BinaryBinary

PreprocPreprocPreprocPreproc GrammarGrammarLinkLink

GrammarGrammarLinkLink

RaiseRaiseErrorsErrorsRaiseRaiseErrorsErrors

Chris Lucas

cut

Static AnalysisThe dev process with Static Analysis

CodCodee CompileCompileCompileCompile StaticStatic

AnalysisAnalysisStaticStatic

AnalysisAnalysis

AnalyzeAnalyzeCodeCode

AnalyzeAnalyzeCodeCode

CheckCheckRulesRulesCheckCheckRulesRules

RaiseRaiseErrorsErrorsRaiseRaiseErrorsErrors

BinarBinaryy

Chris Lucas

cut

Static AnalysisManaged Code Example

static voidstatic void Initialize( Initialize(stringstring id) id) {{ if if (id == "") (id == "") returnreturn;; trytry {{ UtilitiesUtilities.OpenUserAccount(id);.OpenUserAccount(id); }} catchcatch((ExceptionException ex) ex) {{ // user must not exist// user must not exist UtilitiesUtilities.CleanAccountState(id);.CleanAccountState(id); Console.WriteLine(ex);Console.WriteLine(ex); } } } }

CompilesCompilesStatic Analysis Static Analysis gives warnings gives warnings

Static AnalysisManaged Code Examplestatic voidstatic void Initialize( Initialize(stringstring id) id) {{ if if (id == "") (id == "") returnreturn;; trytry {{ UtilitiesUtilities.OpenUserAccount(id);.OpenUserAccount(id); }} catchcatch((ExceptionException ex) ex) {{ // user must not exist// user must not exist UtilitiesUtilities.CleanAccountState(id);.CleanAccountState(id); Console.WriteLine(ex);Console.WriteLine(ex); } } } } e:\Logger.cs(39): 'Logger.Initialize(System.String)#System.Void' e:\Logger.cs(39): 'Logger.Initialize(System.String)#System.Void' Modify the following catch clauses in 'Logger.Initialize' Modify the following catch clauses in 'Logger.Initialize' by catching a more specific exception type or rethrowing the by catching a more specific exception type or rethrowing the exception ' catch(Exception) {...}' exception ' catch(Exception) {...}'

e:\logger.cs(36): 'Logger.Initialize(System.String)#System.Void' e:\logger.cs(36): 'Logger.Initialize(System.String)#System.Void' Replace all calls to "".Equals or String.Empty.Equals in Replace all calls to "".Equals or String.Empty.Equals in 'Logger.Initialize(System.String)' with checks for Length == 0. 'Logger.Initialize(System.String)' with checks for Length == 0.

Static AnalysisManaged Code Examplestatic voidstatic void Initialize( Initialize(stringstring id) id) {{ if if (id.Length == 0)(id.Length == 0) { throw newthrow new ArgumentExeceptionArgumentExeception(“Bad A…”);(“Bad A…”); } trytry {{ UtilitiesUtilities.OpenUserAccount(id);.OpenUserAccount(id); }} catchcatch((ExceptionException ex) ex) {{ // user must not exist// user must not exist UtilitiesUtilities.CleanAccountState(id);.CleanAccountState(id); throw throw ex;ex; } } } }

Improve myImprove mycodecode

Static AnalysisStatic Analysis

ProfilingProfiling

Profiling

• Bad performance is expensive● Customer perception● Hardware● Investigation

ProfilingInstrumentation

mainmain AMethodAMethod BMethodBMethod

main main 001001AMethodAMethod 00230023BMethodBMethod 00980098…….. .. 01450145

• Adds instructions to your code to monitor

ProfilingSampling

mainmain AMethodAMethod BMethodBMethod

• Analyze without changing your app

main 3main 3AMethod 6AMethod 6

BMethod 9BMethod 9

22

Profiler

• Able to do both managed and unmanaged

• Able to do both instrumentation and sampling

• Used by many internal teams and on customer engagements

ProfilingProfiling

Unit TestingUnit Testing

Unit Testing• Unit tests are tests written by

programmers to check the functionality of production code● Always automated● Generally one-to-one mapping between

Unit Test and production code

• Typical problems:● Too much overhead for everyone to learn● Too tedious● Isn’t effective enough to warrant the cost● Not process agnostic

Writing a Unit Test made Easy

• Low overhead, very few moving parts• We do provide simple helper APIs and

easy to understand attributes• We don’t force inheritance, interfaces,

etc

Unit Test Code Generation

• Human error sometimes accounts for missed methods and classes

• Code Generation can develop the skeleton of your unit tests● Namespaces● Classes● Methods

• Generated code fails with result ‘Inconclusive’

Unit TestingUnit Testing

Test Driven DevelopmentTest Driven Development

Test Driven Development

• Think about the code before you write it, have a failing test before you write product code

• The Test Case is a specification● User Scenarios● Business Cases● Minimal bar, boundary conditions,

performance, etc• Infrastructure can be painful

● Build Errors● Managed your failures

Test Driven Test Driven DevelopmentDevelopment

Code CoverageCode Coverage

Code Coverage

• Test Authoring isn’t about quantity, it’s about quality

• Code coverage helps you monitor your tests effectiveness● Team Members can analyze results at a

high-level● Can also analyze source for specific

missed methods and branches

Code Coverage

void PurchaseItem(int itemID)

{

if (itemID == 0) {

throw new Exception();

} else

{ ProcessOrder(itemID);

}

}

UnitTest()

{

PurchaseItem(1);

}

Code CoverageCode Coverage

Team Developer and Team Developer and Team Foundation ServerTeam Foundation Server


cess

an

d A

rch

itect

ure

Pro

cess

an

d A

rch

itect

ure

G

uid

an

ceG

uid

an

ce

Vis

ual S

tud

io In

du

stry

V

isu

al S

tud

io In

du

stry

Part

ners

Part

ners


Visual Studio

Team Architect


Code Profiler

Unit Testing

Code Coverage




Class Designer

Load Testing

Manual Testing




Deployment Designer

Visual Studio


Team TestDynamic Code Analyzer


Code Profiler

Unit Testing

Code Coverage

Change Management

Work Item Tracking

Reporting

Project Site

Visual Studio

Team Foundation Integration Services

Project ManagementBig Build


cess

an

d A

rch

itect

ure

Pro

cess

an

d A

rch

itect

ure

G

uid

an

ceG

uid

an

ce

Vis

ual S

tud

io In

du

stry

V

isu

al S

tud

io In

du

stry

Part

ners

Part

ners

Change Management

Work Item Tracking

Reporting

Project Site

Integration Services

Project Management


Visual Studio

Team Architect


Code Profiler

Unit Testing

Code Coverage




Load Testing

Manual Testing




Deployment Designer

Visual Studio


Team Test

Change Management

Work Item Tracking

ReportingVisual Studio

Team Foundation

Class Designer

Big Build

Roles Addressed by Team System

Project Manager

Architect Developer Tester

Change Management

Work Item Tracking

Reporting

Project Portal

Visual StudioTeam Foundation

Integration Services

Project Management

Visual Visual Studio Studio IDEIDE

Work Item Integration

• Project Management can assign work items and track across lifecycle of the project

• Developers and Testers can assign defects and these integrate with PM work items

• Artifacts (tests, product modules, source code) can be associated with Work Items

Reviews Find / Fix Rates…

..and Daily Trends

…and Bugs Against Code Churn and Testing Activity

Team Developer

• Static Analysis• Profiling • Unit Testing• Code Coverage• AND MORE!!!• Integrated with TFS

● Source Code Control● Project Management● Bug Tracking● Reports

• All fully functional from the VSTS 2005

building robust and reliable software jason anderson, microsoft [email protected]

Documents