quality on submit - romaniatesting.ro · quality on submit asaf saar, sap ag area product owner cpo...

Quality on Submit Asaf Saar, SAP AG Area Product Owner CPO I2M Software Engineering Romanian Testing Conference 2013 – 2nd Edition, May 16, 2013

©  2013 SAP AG. All rights reserved. 2

About me

APO of SAP CPO Software Engineering.

17 years of Software Development, Quality and Testing experience.

Lean, agile, automation and testing manager, trainer and coach.

Email: [email protected]

Twitter: @saarasaf

LinkedIn: http://www.linkedin.com/in/asafsaar

Xing: https://www.xing.com/profile/Asaf_Saar


Agenda

Continuous Integration � Definition and goals � Motivation � Comprehensive Continuous Integration Scenario

Quality on Submit � Overview � Demo � Process Flow �  Transparency


Agenda




Continuous Integration – Definition and goals

“A key software development practice where members of a team integrate their work frequently”

Martin Fowler, Continuous Integration: http://martinfowler.com/articles/continuousIntegration.html



Continuous Integration is all about reducing the risk by providing fast feedback to the team


Continuous Integration – Motivation

Motivation?


Continuous Integration – Motivation

Why are there no updated systems?

Who broke the build?

Who broke the runtime?

How many changes do we have today?

Did Adrian already checked-in his stuff?

What is the status of our product today? Now?

Why do we fix static code issues right before the release?


Comprehensive Continuous Integration Scenario

What makes a Comprehensive Continuous Integration Scenario?



• Fully automated process

• Applying Agile Software Engineering methods

•  Integration testing after each submit

• Run Tests in stages

•  Instant transparency to the team

ET

System Tests

Integration Tests

Component Tests

Unit Tests 100%

10%

Cov

erag

e

Autom

ated M

anual



How do we implement a Comprehensive Continuous Integration Scenario?


Agenda




Quality on Submit Continuous Integration in practice

Quality on Submit is a software quality process which provides qualification of an end to end product scenario using various testing types

The process delivers instant transparency for the quality of each new code check-in, supporting the implementation team to deliver on quality

Developer CI Server

Source Change

Execute automation

Listen

Build

Deploy

Integrate

Scan

Test

HANA DB or SCM Server

Aggregate Results

Finish Run

Employee Dashboard

email

Pu

bli

sh R

esu

lts

Jira

TWB GTP

Show Status

*SAP Patent pending application title "Quality on Submit Process“ SAP Ref: 2010p00489us


Quality on Submit process is implemented using a customized plug-able “process steps” engine

Submit Build Deploy Unit Test Code Quality Notify

Load Testing

Integration Testing

Security Scanning

Quality on Submit Process flow


Quality on Submit Continuous Integration in practice

Demo


Build

SCM Server

Jenkins SCM Plugin (P4, GIT, etc.) Listen to the SCM and trigger the QOS process

Compilation starts (Maven / Ant / etc.)

Build Deploy Unit Test

API Tests

JS Tests Sonar Fortify UI DB

Save Notify Gamify


Deploy

REF System

A dedicated reference system is deployed with the new build results.

This system is always updated to the latest code state


API Tests


Save Notify Gamify

Continuously Deployed


Unit Tests

All Unit tests are detached from the build process

Supports any kind of unit tests like NUnit, JUnit, etc.

Reports provided via Jenkins plugins like surefire


API Tests


Save Notify Gamify


API Tests

Run on the deployed system

Supports various of methods like JMeter, soapUI etc.

Reports provided via Jenkins plugins


API Tests


Save Notify Gamify


Java Script Tests

Support JSLint for JS Static Code Analysis and JsUnit tests like Jasmine.

Reports provided via Jenkins plugins


API Tests


Save Notify Gamify


Sonar

Open source static code analysis platform

Rich HTML dashboards

Numerous plug-ins

Violations, components, hotspots, time machine and a lot more…

Build Deploy Unit

Test API

Tests JS

Tests Sonar Fortify UI DB Save Notify Gamify


Fortify

Licensed security test tool by HP

Scan the current static code and detects security violations

Provide rich HTML dashboards


API Tests


Save Notify Gamify


UI Automation

Set of Integration tests executed

Supports various technologies like Selenium, HP QTP, Sikuli and more

Reports provided via Jenkins plugins or by any external provider


API Tests


Save Notify Gamify


Data saved into a DB

All data is saved into a DB

QOS Cycles statistics can be generated

Developers typical behavior can be viewed from this DB

Build Deploy Unit

Test API

Tests JS

Tests Sonar Fortify UI DB Save Notify Gamify


Email Notification

Notification sent to the implementation team Transparency to the implementation and product teams

Elaborated status and links for all phases


API Tests


Save Notify Gamify


Gamification

Optional connection to a Gamification system which calculate phases results and provide ranks and badges

Increase the fun in development

Increase the drive for quality


API Tests


Save Notify Gamify


Product Team Transparency

Daily report sent to the product team Transparency


Quality on Submit

Summary


Quality on Submit End To End Quality

Ø  Instant feedback in 45 minutes max

Ø  Increased team responsibility and quality awareness

Ø  Rewards development for quality

Ø  Makes TDD realistic and drives towards it


Quality on Submit Prerequisites

Split your unit tests from your build

Split your build and make it short

Make your tests in short cycles

Make you tests reliable

Avoid false alarms

Build

Deploy

Integrate

Scan

Test


Reference tools

Jenkins CI: http://jenkins-ci.org

Sonar: http://www.sonarsource.org

Selenium: http://www.seleniumhq.org/

Jmeter: http://jmeter.apache.org/

Junit: http://junit.org/

JSLint: http://www.jslint.com/

Asaf Saar, SAP AG Area Product Owner ECF Software Engineering [email protected]

Thank You!


No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft, Windows, Excel, Outlook, PowerPoint, Silverlight, and Visual Studio are registered trademarks of Microsoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, z10, z/VM, z/OS, OS/390, zEnterprise, PowerVM, Power Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER, PowerHA, pureScale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, AIX, Intelligent Miner, WebSphere, Tivoli, Informix, and Smarter Planet are trademarks or registered trademarks of IBM Corporation.

Linux is the registered trademark of Linus Torvalds in the United States and other countries.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are trademarks or registered trademarks of Adobe Systems Incorporated in the United States and other countries.

Oracle and Java are registered trademarks of Oracle and its affiliates.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems Inc.

HTML, XML, XHTML, and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

Apple, App Store, iBooks, iPad, iPhone, iPhoto, iPod, iTunes, Multi-Touch, Objective-C, Retina, Safari, Siri, and Xcode are trademarks or registered trademarks of Apple Inc.

IOS is a registered trademark of Cisco Systems Inc.

RIM, BlackBerry, BBM, BlackBerry Curve, BlackBerry Bold, BlackBerry Pearl, BlackBerry Torch, BlackBerry Storm, BlackBerry Storm2, BlackBerry PlayBook, and BlackBerry App World are trademarks or registered trademarks of Research in Motion Limited.

© 2013 SAP AG. All rights reserved.

Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps, Google Mobile Ads, Google Mobile Updater, Google Mobile, Google Store, Google Sync, Google Updater, Google Voice, Google Mail, Gmail, YouTube, Dalvik and Android are trademarks or registered trademarks of Google Inc.

INTERMEC is a registered trademark of Intermec Technologies Corporation.

Wi-Fi is a registered trademark of Wi-Fi Alliance.

Bluetooth is a registered trademark of Bluetooth SIG Inc.

Motorola is a registered trademark of Motorola Trademark Holdings LLC.

Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase Inc. Sybase is an SAP company.

Crossgate, m@gic EDDY, B2B 360°, and B2B 360° Services are registered trademarks of Crossgate AG in Germany and other countries. Crossgate is an SAP company.

All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG.

quality on submit - romaniatesting.ro · quality on submit asaf saar, sap ag area product owner cpo...

Documents