enabling automated integration testing of cloud application services in virtualized environments

NORTH DAKOTA STATE UNIVERSITY DEPARTMENT OF COMPUTER SCIENCE

© NDSU SOFTWARE TESTING RESEARCH GROUP

Enabling Automated Integration Testing of Cloud Application

Services in Virtualized EnvironmentsTariq M. King, Annaji S. Ganti, and David

Froslie

© NDSU SOFTWARE TESTING RESEARCH GROUP 2

Outline Introduction Background Research Problem Approach Prototype Discussion Related Work Conclusion & Future Work


Cloud provides ubiquitous, on-demand access to computing resources “as services” via the Internet

Operational Perspective Availability, Scalability, Efficiency, …

Development Perspective Reusable Components, Info. Hiding Highly Complex Local Infrastructure

Testing Perspective?

Introduction

3


Cloud poses several new testing challenges but also brings a number of benefits to software testing

Challenges of Testing Cloud Applications High Complexity, Remoteness, Information Hiding,

Autonomy, High Performance and Dependability…

Benefits of Software Testing In The Cloud (STITC) Computational Power and Storage Virtualization

Theme: Use Benefits to Overcome Challenges

Introduction

4

MOTIVATION


“Enabling Automated Integration Testing of Cloud Application Services in Virtualized Environments”

STITC is an emerging discipline with potential to significantly change how we do testing

STITC 2010 Workshop focused on three key areas: Testing IN the Cloud – leveraging the cloud

for test execution and testing in a virtualized environment

Testing OF the Cloud – validating applications that are hosted and deployed in the cloud

Testing TO the Cloud – moving the testing process and other assets to the cloud

Introduction RESEARCH POSITION


“Services” in cloud computing typically fall into three categories:

Software-as-a-Service | Google Docs, Office Web Apps

Platform-as-a-Service | Google App Engine, Windows Azure

Infrastructure-as-a-Service | Amazon EC2, Rackspace

Background CLOUD COMPUTING


Many cloud platforms offer Virtualization support Creating abstract computing resources from

more powerful physical resources

Hosted Architecture

Background VIRTUALIZATION

Hypervisor Architecture


To narrow the research scope, we analyze a specific cloud application development scenario

One host (Provider B) is developing an application service that will extend the functionality of a service offered remotely by another host (Provider A)

Research Problem SCOPE


Even with its limited scope, the scenario reveals many integration testing issues and challenges: Difficult for Provider B to Setup Some Tests

for his application since he does not have full control and observation of Service A. Possible Solutions?

Developing Accurate Stubs is Hard since Provider B has no knowledge of Service A’s implementation

Provider B Limited to Black Box Testing of those parts of his application implemented by Service A

Service B can Change w/o Provider B’s Knowledge i.e., Update to Service A Need to Retest Service B

Research Problem ISSUES


Our approach to the research problem introduces the notion of Test Support as-a-Service (TSaaS)

Prior to deploying Service A, Provider A would have tested it, which typically involves creating: Test Artifacts – Automated Test Drivers and

Stubs A Test Bed – Hardware/Software

Environment

TSaaS seeks to reuse such existing test automation for the development and delivery of a set of test support services for collaborating partners

Approach


Provider B has full access to a test copy of Service A Test hooks, instrumentation, and scaffolding

reuse Test data (non-confidential) can be made

available Testing does not interrupt service in

production

Concerns: Maintenance, Scalability, Security

Approach OVERVIEW

Provider A exposes TSaaS to Provider B


Hypervisor-Based Virtualization and Secure Access

Approach ARCHITECTURE


Approach CONTROL FLOW

Control Flow When Realizing Test Support Services


Designed with goal of making its components easily deployable to Windows Azure®

Web & Worker Roles (ASP.NET, IIS7, WCF Services)

VM Role (Windows 2008 R2 Server Image)

Application Domain: Credit Reporting/Banking

Prototype PROOF OF CONCEPT


The primary reason for developing the prototype was to investigate the feasibility of implementing TSaaS

Successes Able to quickly develop and expose a cross

section of test support operations for the credit service by reusing its existing test artifacts and tooling Test authoring and execution, code

coverage, error reporting, and server-side diagnostics

Facilitates independently setting up preconditions, applying inputs, making assertions, doing cleanup

Discussion


Technical Challenges Locating and Integrating Required

Technologies Combines programming knowledge, test

automation skills, and runtime virtualization expertise

Configuring Virtual Hard Disk Images of Servers Enable Auto-Login, Install Security Patches,

Disable Reboot after Auto-Update, Set User Privileges

Emphasized the importance and need for VM tools for manipulating and updating these images

Discussion ACCIDENTAL DIFFICULTIES


Key Research Challenge Provision of standardized interfaces and

guidelines for performing testing in the cloud and developing online test support services for the cloud.

Limitations Lack of formalized evaluation of the

approach Narrow scope of research problem and no

evidence of generality of the approach Feasible to Implement Practical

Discussion


Formal modeling and model-based testing criteria for cloud applications (Chan 2009)

Model-driven approach to security testing of cloud applications (Zech 2011)

Industry challenges for runtime integration and testing for software services (Greiler 2009)

Many works on testing and testability of SOA(Mei 2008, O’Brien 2007, Tsai 2006)

Related Work


Presented an approach to support integration testing of cloud applications using virtualized environments – Test Support as-a-Service (TSaaS)

Described implementation of a TSaaS Prototype Provides some evidence that approach is

feasible

Future work calls for Evaluation and Extension Azure Experiments, Alternative

Implementation, Standardization, Expand Problem Scope, Benefits Virtualization Benefits/Implications, Performance

Building Self-Testing into the Cloud (STITC 2011)

Conclusion and Future Work


Acknowledgements Neha Kale, MS Student (NDSU)

Dr. Scott Tilley, Professor (Florida Tech), and the participants of STITC 2010

Alisson Sol, Knowledge Engineer (Microsoft)

and the participants of 2011 Microsoft Fargo Engineering Day Excellence Expo

CASCON 2011 Reviewers


Thank You!Questions?

¿Preguntas?問題Sawwalвопросы質問domandeερωτήσεις

enabling automated integration testing of cloud application services in virtualized environments

Documents

cloud testing

cloud computing

testing process

cloud introduction5research

development perspective

new testing challenges

cloud stitccomputational

software design engineer