Slide 1

CSCE 181 Test Automation at Schlumberger Glen Accardo, HTC Test Automation Lead John Wanke, HQ Test Automation Engineer

Slide 2

2 Mandatory Attribution Slide (this title should not display) 2011 Schlumberger. All rights reserved. An asterisk is used throughout this presentation to denote a mark of Schlumberger. Other company, product, and service names are the properties of their respective owners.

Slide 3

Agenda Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

Slide 4

Glen Accardo Started at Schlumberger in 2005 Tested drilling, real-time, and production applications Currently I manage the test automation function at the Houston Technology Center

Slide 5

Glen Accardo Studied at University of Southwestern Louisiana Computer Science, Mathematics, English Held the following positions before Schlumberger Technical writer Web master/application developer Developer/Tester for database performance monitoring/tuning, and backup/recovery http://www.linkedin.com/pub/glen-accardo/0/905/904

Slide 6

Agenda Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

Slide 7

What is Schlumberger? The leading provider of technology and project management to the upstream oil and gas industry for over 80 years Over 110,000 employees, 140 nationalities in 80 countries 25 research & engineering facilities worldwide-2010 invested $919 million in R&D The only true multinational corporation in the composition of its workforce, management team and Board of Directors

Slide 8

Slide 9

A History of Success for more than 80 years

Slide 10

Software Products & Services Software Geoscience Petrophysics Basin Modeling Reservoir Engineering Drilling Production Economics Information Management Open Development Platform Services Collaboration Centers Production Operation Centers Rapid Field Screening Integrated Asset Modeling Real-time Drilling Reserves Management Corporate Data Management National Data Centers Global connectivity Petrotechnical Computing

Slide 11

Software Products Ocean - Open development platform GeoFrame - Geoscience interpretation and visualization Petrel - Seismic to simulation workflows Studio E&P Knowledge Environment Techlog - Petrophysical analysis ECLIPSE - Industry reference reservoir simulation Avocet, OFM, PIPESIM - Production engineering and operations management Merak - Economics, reserves management and capital planning Information Management - Corporate and project data management, data quality management, context aware data search

Slide 12

SIS Product Centers

Slide 13

Agenda Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

Slide 14

Anyone Recognize This Phrase? a sound preparation in science, mathematics, English, statistics and computing. Program Overview The computer science curriculum is designed to prepare students to enter the rapidly expanding computer field. It is based upon the Institute of Electrical and Electronic Engineer Computer Society and the Association for Computing Machinery recommendations for curricula and courses. The four- year undergraduate curriculum in computer science includes a sound preparation in science, mathematics, English, statistics and computing. Students majoring in computer science at Texas A&M will take a 12-hour area of concentration in another department. Electives are available in the areas of: algorithms, languages and computability, software systems, networking, computer systems and architecture, artificial intelligence and cognitive modeling, vision, graphics and robotics and computational science and engineering. Advanced undergraduates may take graduate courses in these areas.

Slide 15

Why is English in the List? Nearly all jobs require Excellent communications skills Employers will probably see your writing before your programminglaziness or ineptitude in one will be applied to the other Not just for technical writers Not just for a job interview Clear writing will distinguish you from people with equal technical skills

Slide 16

Agenda Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

Slide 17

What is Testing? Testing is an infinite process of comparing the invisible to the ambiguous in order to avoid the unthinkable happening to the anonymous. - James Bach

Slide 18

Problem Number 1 Development Effort Testing Effort Incremental Cumulative Which Leads To: - Risks for customers - Risks for product team - Increased support cost Debate the finish criteria: - Remove features - Extend deadlines - Add resources Take short cuts: - Dont test features - Dont test configurations - Dont do non-functional tests

Slide 19

Solution: Add Test Automation Test Automation Effort What if we eliminate repetitive tests? Development Effort We can refocus manual test efforts on important, new testing. Manual Test Effort

Slide 20

What is Test Automation? Software to test software Mimic behavior of a user or application in an environment Can be part of code baseline (nUnit, Junit) Can be done at UI level (TestComplete)TestComplete Can be hybrid

Slide 21

What Must Automated Tests Do? Mimic behavior of a user or application in an environment Validate the results of this behavior Maintain or improve upon standards of other forms of testing.

Slide 22

What is Test Automation Not? Not record and playback Not a cheap alternative to other methods of testing

Slide 23

Example The TestComplete version of Hello World.TestComplete

Slide 24

Impact? Application exists (is installed?) Application starts without error Application accepts input without error Application saves file without error Application exits without error

Slide 25

Basic Metrics MetricValue Time to Develop15 minutes Time to run16 seconds Time to run manually2 minutes MetricValue Performance Improvement86% Can run 7.5x as often CostCost of computer for 16 seconds vs. Cost of person for 2 minutes ExpandabilityCan run simultaneously on any number of configurations RepeatabilityNear perfect

Slide 26

Problem Number 2 Question: Should add feature X? Answer: Hmmm. Question: Should we support configuration X? Answer: Yes.

Slide 27

Testing Effort Grows on Multiple Dimensions Features Configurations Data Sets

Slide 28

Two Part Solution All Pairs Testing http://en.wikipedia.org/wiki/All-pairs_testing http://en.wikipedia.org/wiki/All-pairs_testing Run automated tests in parallel

Slide 29

WebCam of HTC Automation

Slide 30

Impact ProductAutomated TestsEquivalent Manual Tests Studio20 hours * 4 platforms8 person weeks Matira5 hours * 3 platforms4 person weeks Tupai3 hours * 9 platforms5 person weeks PIPESIM40 hours * 5 platforms????? Seabed4 platforms * 1 hour1 person week OFMTBD MerakTBD MEPTBD IAMTBD

Slide 31

Problem Number 3

Slide 32

Agenda Introduction Overview of Schlumberger A footnote about your degree program What is automated testing, and why do it? Challenges in test automation

Slide 33

John Wanke Started at Schlumberger in 2010 Tested economics, asset management, and enterprise planning applications Currently I lead the test automation effort for the Production Solutions group (services-based products)

Slide 34

John Wanke Studied at Wichita State University Electrical Engineering, Mathematics Held the following positions before Schlumberger Lead Systems Analyst / Software Engineer Lab Systems Administrator Web master/application developer I/T Manager Software Quality Assurance Engineer/Developer for message queueing administration and monitoring http://www.linkedin.com/in/johnwanke

Slide 35

Management, Marketing, and Machiavelli Well-known Established Easily understood Simple operation Alien and foreign Newcomer Complex Requires skill to use

Slide 36

Challenges of Test Automation: Manage Expectations MetricValue Time to Develop15 minutes Time to run16 seconds Time to run manually2 minutes RealityMeaning Development SkillsMuch more than hello world I/T skillsComputers dont magically appear and operate Application KnowledgeNot all applications are == notepad Automation tool skillsThe application doesnt run itself, does it? Testing skillsWhat can be tested? How? CollaborationDev aware of how to test. Tester must show results.

Slide 37

Merak Enterprise Planning (MEP) Solution Overview

Slide 38

MEP Product Architecture

Slide 39

The Technical Challenge The challenge you face is gaining automated / deterministic control of your test environment. -- Jonathan Lee

Slide 40

The Technical Challenge, Part 1: Known Test Platforms

Slide 41

Merak Enterprise Planning Test Environment

Slide 42

The Technical Challenge, Part 2: Known Testware Test Automation Effort Development Effort Manual Test Effort Software written to test software Subject to same controls as any other software asset Versioned to be in sync with app

Slide 43

The Technical Challenge, Part 3: Find a Tester or Make One

Slide 44

Q & A

Slide 45

Automation Strategy Set evaluation criteria Examine test cases and test scripts Adjust testing process Involve automation users Train people

Slide 46

Infrastructure Ownership Defines processes and disciplines Helps guarantee adherence Promotes acceptance Eliminates duplication of effort Encourages code reuse and modularity Prevents corruption due to unsolicited modification