Introduction To Clarity

What is Clarity?

Clarity is a Test Automation Framework to

make automation

● Eazy

● Organized

● Relevant

● Optimized

Why Clarity?

● To automate platform-wide scenarios.

● To execute tests against stratos and private clouds.

● To adopt tooling support to automation

○ Selenium

○ SoapUI

○ Jmeter

○ Ravana

● Reporting and keeping historical records.

● Coverage.

WSO2 Platform Scenario

More On Clarity?Automate scenarios with multiple products with multiple

users/ tenants.

● Execute tests using different DBs, OS, browsers and

app servers.

● Support for webcontext root, different port changes,

host name bindings.

● Testing Clustered environments.

● Run tests against up and running server.

Unit Vs Integration TestsUnit Tests:

● Focused on defining (not testing) the behavior of a particular class, and that class only.

● Fast (less than 0.1 second per test)● No external implementation dependencies (filesystem, database, web

services, etc.). All dependencies are faked for the test context.● Can easily be parallelized, since each test is atomic

Integration Tests:● Focused on verifying the integration of one or more components

together.● May have external dependencies. (in fact, it is likely testing the

integration with this dependency)● A test that takes longer than a unit Test should (longer than 0.1

seconds per test)

Integration Tests Vs Platform TestsIntegration Tests - with in wso2 product

● Executes on product pack created by build.● Not depending on other product environments.● Independent.● Lightweight.● Covers only scenarios related to features on individual product.

Platform Tests:● Can run on a configured ( With mounts , Clusters ...) product packs

and already up and running server. ● Can generate scenarios based on multiple product environments.● Can execute load and UI tests

Architecture Overview

Reference:- http://www.ibm.com/developerworks/java/library/j-testng/

● Follows TestNG Annotations bases

Execution Structure.

● Operations based on TestNG Listeners.

○ Server startup/Shutdown.

○ Emma instrumentation, Report Generation And Cleaning.

○ Artifact Deployment /Artifact cleaning.

○ Tennent Creation and Registering.

○ Report Generation.

● Test Optimized API.

● Utility Classes to Support test cases.

Execution Template for ClarityExecution Started

@BeforeSuite@BeforeTest

@AfterTest

@AfterSuite

Execution Closed

@BeforeClass

@BeforeGroups

@BeforeMethod

@AfterMethod

@AfterGroups

@AfterClass

TestNg Listeners● Listeners are interfaces allows to modify TestNG's

behavior.

● Listeners are binned to a testNg execution.

● Listeners implement the interface org.testng.ITestListener○ IExecutionListener

■ Triggers at start and end of any execution.■ ISuiteListener.

○■ Triggers at suite start and end.ITestListener■ Triggers at Test start, Finish, Failure, Skip and partial failure.

● Reporters implement the interface org.testng.IReporter ○ Notified when all the suites have been run by TestNG. ○ The IReporter instance receives a summary of entire test run

Usage of Listeners in Clarity● PlatformExecutionManager (IExecutionListener)

○ onExecutionStart()

■ Emma instrumentations

■ Server start

○ onExecutionFinish()

■ Server Sutdown

■ Emma report generation

● PlatformSuiteManager (ISuiteListener)

○ Set environment properties ex- Key Store Paths

○ Populate Users.

Usage of Listeners in Clarity cont.d

● PlatformTestManager (ITestListener)

○ On Start

■ Artifact Deployment

○ On Finish

■ Artifact Clean up

● PlatformReportManager (IReporter)

○ Generate TestNg Report

○ Generate sure-fire report

○ Export data for Dashboard

Clarity Architecture

Artifact Repository|-- AS| |-- aar| | |-- Axis2 Service.aar| |-- car| | `-- AxisCApp-1.0.0.car|-- BPS| `-- bpel| |-- CleanUpTest1.zip| |-- BRS| |-- aar| |-- rsl| `-- ruleservice| `-- GreetingService.aar|-- DSS| |-- dbs| | |-- csv|-- ESB| |-- car| | |-- DemoProxy-1.0.0.car| |-- endpointlookup.xml|-- GREG| |-- artistRegistry.gar|-- GS| `-- gadget| `-- HelloWorldGadget.xml|-- IS| |-- IIA001Policy.xml`-- MS `-- jszip `-- schemaTest1.zip

● Location for keeping all artifacts

needs for your Test

● Refers by automated Artifact

deployment process

● Manages my file structure

depending on product and artifact

type.

Clarity Properties

# Testing server details#---------------------------------------------------------------------------------------------------stratos.test=falseport.enable=truecarbon.web.context.enable=falseservice.deployment.delay=30000cluster.enable=falseproduct.list=GREGdeployment.framework.enable=falsedeployment.framework.home=/home/krishantha/automation/deployment-framework

coverage.enable=falsegreg.carbon.home=/home/dharshana/latestPack/gregPack2/wso2greg-4.0.0-SNAPSHOT

as.host.name=localhostas.http.port=9764as.https.port=9444#as.webContext.root

● File Location <FRAMEWORK_HOME>/core/org.wso2.automation.platform.core/src/main/resources/framework.properties

● Contains all information

about your environment.

● To configure WSO2 clarity

for your environment and

configure package details

Manage Tenants

"0","admin","admin123""1","admin123@wso2manualQA0002.org","admin123""2","admin123@wso2manualQA0004.org","admin123""3","admin123@wso2manualQA0006.org","admin123""4","admin123@manualQA0001.org","admin123""5","admin123@krishantha.hhh","admin123""6","admin123@wso2autoQA0008.org","admin123""7","admin123@wso2autoQA0009.org","admin123""8","admin123@wso2autoQA0010.org","admin123""9","admin123@wso2autoQA0011.org","admin123""10","admin123@wso2autoQA0012.org","admin123""11","admin123@wso2autoQA0013.org","admin123""12","admin123@wso2autoQA0014.org","admin123"

● File Location● <FRAMEWORK_HOME>/core/org.wso2.automation.platform.

core/src/main/resources/tenantList.csv● <FRAMEWORK_HOME>/core/org.wso2.automation.platform.

core/src/main/resources/usertList.csv

Automated Artifact Deployment

<?xml version="1.0"?><scenarios> <scenario testName="org.wso2.automation.common.test.as.service.AARServiceSpaceInFileNameTest"> <products> <product name="AS"> <artifacts userId="1"> <artifact name="Axis2 Service.aar" type="aar"/> </artifacts> </product> </products> </scenario>

Write First Clarity Test - Demo

Reporting● Includes TestNg reports ,Junit

Reports ( Sure Fire report), And

Emma Coverage report

● Directories containing separate

results for each suite ( Directory

name is as same as the suite name).

● Junit Reports ( Surefire reports).

● index.html (Dashboard for executed

test).

|-- reports | |-- BPSStructuredSuite | | |-- classes.html | | |-- FlowClient.html | | |-- FlowClient.properties | | |-- FlowClient.xml | | |-- ForEachClient.html | | |-- ForEachClient.properties | | |-- ForEachClient.xml | |-- index.html | |-- junitreports | | |-- TEST-org.wso2.automation.common.test.bps.mgts.BpelStructAct_FlowClient.xml | | |-- TEST-org.wso2.automation.common.test.bps.mgtst.BpelStructAct_forEachClient.xml

Reference

OT article

http://wso2.org/library/knowledge-base/2012/06/configure-execute-tests-clarity

TestNg documenation

http://testng.org/doc/documentation-main.html

SVN Location https://svn.wso2.org/repos/wso2/carbon/platform/trunk/platform-integration/system-test-framework

Questions?