Download - Automation for developers
Automation For Developers
● We are super developers and we don't make mistakes.
● I never do changes to my code.
● I am the life time owner of my code.
● I am the only person who use each component I build.
● We have telepathic.
● My code runs on each platform
○ Its platform vendor’s responsibility to make
platforms unique.
What do you think ??????
Who write the code ??
Who works with your code??
●
●
●
●
●
“Quality is free, but only to thosewho are willing to pay heavily for it.”
- Lister, Demarco: “Peopleware”
●
●
●
●
●
●
Testing Good Bad And Ugly
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
Understanding - Cost of test Wrong way
Understanding - Cost of test Right way
Basic Understanding - Unit Test
● Focused on testing behavior of a
particular class, module or method.
● Smallest part of the application.
● Fast (less than 0.1 second per test)
● No external implementation
dependencies
(filesystem, database, web services, etc.). All
dependencies are faked for the test context.
● Can easily be parallelized, since each test is atomic
Basic Understanding - Integration Test
● Focused on verifying the integration
of one or more components together.
● May have external dependencies.
● A test that takes longer than a unit
Test should
Integration Tests - with in WSO2 ● Executes on product pack created by build.
● Not depending on other product environments.
● Independent.
● Lightweight.
● Covers only scenarios related to features on individual product.
●●
○○○○○○○
●M = E − N + 2P,
where
E = the number of edges of the graph.
N = the number of nodes of the graph.
P = the number of Connected Components
●fi
●
●
●●●●
Jacoco /EclEmma coverage report
Emma coverage report
● Platform scenarios.
● Integration scenarios.
● UI scenarios.
● Real world use cases.
● Performance and Security scenarios.
● Patches provided.
How tests really looks like
Platform - Platform Tests
Cross Platform Tests
Product - Integration Tests
Component - Unit Tests
Component - Unit Tests
Component - Unit Tests
Component - Unit Tests
Product - Integration Tests
Component - Unit Tests
Component - Unit Tests
Component - Unit Tests
Component - Unit Tests
Product - Integration Tests
Component - Unit Tests
Component - Unit Tests
Component - Unit Tests
Component - Unit Tests
Product - Integration Tests
Component - Unit Tests
Component - Unit Tests
Component - Unit Tests
Component - Unit Tests
What got better and bigger
● Lightweight test execution engine.
● More focus on platform wide scenarios.
● More user friendly configuration management.
● Ability to extend TestNG based listener implementation
○ Ability to plugin custom servers like Tomcat, CXF ,ActiveMQ,
Apache FTP and more.
● Ability to come up with your automation story.
Organization Test Automation Framework modules
Engine
Extensions Admin Service API
Core - Utilities
Automation Test
Core Servers External Tools
Extensibility Modules
Configuration
Overall Architecture
Configuration
Support Modules Test Automation Framework
Test
Build platform -Maven
Test Trigger- Surefire
Testing Framework - TestNg
Test Suite
platform
Automation.xml
Coverage
Instrumentation.txt
Pluggable Extensions
TestNg Listener Extension Modules
Test Tooling Extensions
Server Management
Tenant Management
Engine
TestNg Listener Extensions
Configuration Management
Context Management
Extensible module handling
Admin Service API
Reusable core utils
filter.txt
Test Configurations
TestNg.xmlTestNg.xmlTestNg.xml
Flexibilities Provided
● Manage several products inside single test.
● Manage users in several product domains.
● Deployment of artifacts for different products under
different users.
● Configured admin service clients as test oriented API.
● Retrieving test environment configurations easily.
● Running same test in both local and Stratos environments.
● Reporting TestNg and surefire reports for all your tests.
● Tooling support of Selenium and Jmeter.
Technologies for Automation
Test Framework Build Systems
Code Coverage
Extensibility Support
UI Automation
● Like JUnit but not JUnit
● Annotation based Architecture
● Test Grouping
● Extensibility features
○ Listeners
○ Bean Shell
○ Annotation Transformers
● Easiness of managing Test cases.
● Failed test execution (Straight and easy)
●
●
●
○ Trigger out TestNg.
○ Control TestNG flow (Through Surefire TestNg Utils)
○ Generate reports. (HTML, XML and emailable report)
Execution Started@BeforeSuite
@BeforeTest
@AfterTest
@AfterSuite
Execution Closed
@BeforeClass
@BeforeGroups
@BeforeMethod
@AfterMethod
@AfterGroups
@AfterClass
● Listeners are interfaces allows to modify TestNG's
behavior.
● Listeners are binned to a testNg execution.
● Listeners implement the interface org.testng.ITestListener○ IExecutionListener
■ Triggers at start and end of any execution.■ ISuiteListener.
○ ISuiteListener■ Triggers at suite start and end.ITestListener■ Triggers at Test start, Finish, Failure, Skip and partial failure.
● Reporters implement the interface org.testng.IReporter ○ Notified when all the suites have been run by TestNG. ○ The IReporter instance receives a summary of entire test run
●
○ Offline instrumentation
○ On fly instrumentation
●
●
○ LOC coverage
○ Class coverage
○ cyclomatic complexity
●
●
Common Extensions
Automation.xml
Test Suite
Test Suite Extensions
Test Suite Utilities
Test classes
Automation Framework Engine
●
●
●
●
● Includes all utility classes that can play supportive role inside a test.
○ Ex :-
■ Axis2 client.
■ Wire message monitor.
■ Custom server startup scenarios (Axis2, Tomcat, ActiveMQ).
■ Concurrency test scenarios.
■ Rest Clients.
● Provides common methodology for all development
teams and Automation team to maintain a set
of supportive classes without changing the Core.
Questions?
Happy Automation !!