George WilsonGeorge Wilson

Original SoftwareOriginal Software

Exploring why Software Test automation fails or succeeds.

Survey

Before we start……..

Please complete the survey forms handed out

We will collect them and give you a couple of key figures at the end of this session.

We will collate and publish the full survey. You can have a copy.

Thanks

Survey 2 – show of hands….

How many of you have automation tools in your company?

How many have on-going success?

Recent surveys

BCS Sigist (March 2007)Special interest group in software testing

200 attendees

% tried automation80%

% successful10%

Butler Testing Symposium (Feb 2008)

QA managers and test analysts

160 attendees

% tried automation70%

% successful2.5%

0 100

EffortSuccess

Survey Stats – We asked 100 CIOs

What is the perceived importance of SQA in your Organisation?

Nice to Have 25%

Cost of doing business 16%

Not Defined 15%

Fundamental Business process 29%

Strategic Business Imperative 15%

www.origsoft.com

Biggest inhibitor? Scripting code driven

High maintenance burden

Limited audience

Technical specialists

Often shelf-ware

Sub Main Dim Result(50) As Integer Dim i as Integer Dim NewResult as String StartBrowser "http://pandora.ple.blahblah.co.uk/action.pega?f1000=SIGNON&profile=test3", "WindowTag=WEBBrowser" Window SetContext, "WindowTag=WEBBrowser", "" Window WMaximize, "", "" delayfor 3000 Browser SetFrame,"Type=HTMLFrame;HTMLId=__pegMainFrame","" Browser NewPage,"HTMLTitle=PANDORA - TEST1","" Result(1) = EditBoxVP (CompareProperties, "Type=EditBox;Name=f5p1", "VP=Object Properties;Wait=2,30") Result(2) = EditBoxVP (CompareProperties, "Type=EditBox;Name=f6p1", "VP=Object Properties2;Wait=2,30") Result(3) = EditBoxVP (CompareProperties, "Type=EditBox;Name=f8p1", "VP=Object Properties3;Wait=2,30") Result(4) = EditBoxVP (CompareProperties, "Type=EditBox;Name=f9p1", "VP=Object Properties4;Wait=2,30") Result(5) = PushButtonVP (CompareProperties, "Type=PushButton;Name=@ACTION=|ENTR", "VP=Object Properties5;Wait=2,30") Result(6) = PushButtonVP (CompareProperties, "Type=PushButton;Name=@ACTION=|APPR", "VP=Object Properties6;Wait=2,30") Result(7) = PushButtonVP (CompareProperties, "Type=PushButton;Name=@ACTION=|BACK", "VP=Object Properties7;Wait=2,30") For i = 1 to 3 Select Case i Case 1 InputKeys "robir" PushButton Click, "Type=PushButton;Name=@ACTION=|ENTR"

Re-use is key

ProcessKnowledgeResultsData

Automation meaningless without re-use

Case Study 1 – Value for money?

Large US financial organisation

“Our Automated Software Testing requirements are fully satisfied”

$1m spent on ASQ

Automation successfully deployed……2 years in set up with team of 6 persons

Covered 5% of all testing effort

Targeted at stable areas of application only

Largely used for data creation only

Case Study 2

Another large US financial organisation.

Started automation Feb 2008 for a key project due in August.

Delivered the project 3 months early, fully tested.

Calculated the time savings as a cost saving of $270k per annum.

December 2008. Used to put live up to 3 projects at a time, now they put live up to 20 at a time.

Now progressing to tackle UAT also.

Risk Change TriangleC

hang

e

HighChange

High Risk

StableLow Risk

Ris

k

Automationzone

Manualzone

Raise the bar

HighChange

High Risk

StableLow Risk

Automationzone

ManualzoneRaise the

bar

Progress from manual to auto

www.origsoft.com – gwilson@origsoft.com