testing sap: modern methodology
DESCRIPTION
Overview of modern testing methods and tools, with a focus on SAP BI, BW, planning, and EPM projects.TRANSCRIPT
![Page 1: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/1.jpg)
Testing SAP:Modern Testing Methodologies
Ethan Jewett
![Page 2: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/2.jpg)
But first!
What’s the problem?
![Page 3: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/3.jpg)
“Enterprise” project trade-offs
Value
Risk
Duration
Budget
![Page 4: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/4.jpg)
Software development trade-offs
Features
Quality
Time
Budget
![Page 5: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/5.jpg)
Client requests
Build FasterAnd produce more Value
Cheaperwith less Risk
![Page 6: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/6.jpg)
Remember…
Value
Risk
Duration
Budget
![Page 7: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/7.jpg)
Are there hidden assumptions?
Yes.
MethodologyPeople and Culture
Technology
![Page 8: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/8.jpg)
Any other assumptions?Project Preparation
Business Blueprint
Project Realization
Go-live and Support
Technical Design
Technical Build Test Fix Transport Go Live
Write test scripts
Write and run unit test
Integration & Regression
TestTraining Live
Support
![Page 9: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/9.jpg)
The topic
Over the next hour and a half, we’ll talk about how testing methodologies, techniques, and technologies can help us
Change these underlying assumptions.Enable modern project methodologies.
Control the basic project trade-off.
![Page 10: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/10.jpg)
The focusProject Preparation
Business Blueprint
Project Realization
Go-live and Support
Technical Design
Technical Build Test Fix Transport Go Live
Write test scripts
Write and run unit test
Integration & Regression
TestTraining Live
Support
![Page 11: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/11.jpg)
The goal
More valueFaster
With less risk
(and more fun)
![Page 12: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/12.jpg)
For example
![Page 13: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/13.jpg)
V-Model methodology
http://en.wikipedia.org/wiki/File:V-model.JPG
![Page 14: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/14.jpg)
More parallel, less repetition
http://en.wikipedia.org/wiki/File:V-model.JPG
![Page 16: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/16.jpg)
More iterative
http://en.wikipedia.org/wiki/File:V-model.JPG
![Page 17: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/17.jpg)
How do we get there?
![Page 18: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/18.jpg)
Modern methodology
• Modern quality assurance and testing– Leading practice in custom development, open
source, and agile projects• Test driven development (TDD), behavior driven
development (BDD), readable tests, executable requirements, continuous integration
– Overview of the SAP test automation technologies– Road we're going to take through testing SAP
![Page 19: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/19.jpg)
Leading Practice
• Context• Questions and problems• Approaches• Tools
![Page 20: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/20.jpg)
Leading Practice
• Context• Questions and problems• Approaches• Tools
![Page 21: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/21.jpg)
Context
The context of modern software development is above all open, agile and decentralized.
Open source projects are usually the furthest along this path, and as such these projects use (or are forced to invent) tools that fit their processes.
![Page 22: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/22.jpg)
Openness
Requirements, specifications, issues reports, test cases, source code, and project tools and metrics are readily open and available to all participants in the project.
(Not necessarily the same as “open source”.)
Open Collaboration within Corporations Using Software Forges - http://www.riehle.org/publications/2009/open-collaboration-within-corporations-using-software-forges
![Page 23: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/23.jpg)
Agility
The values of the Agile Manifesto:– Individuals and interactions over processes and
tools– Working software over comprehensive
documentation– Customer collaboration over contract negotiation– Responding to change over following a plan
http://agilemanifesto.org/
![Page 24: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/24.jpg)
Decentralization
Open-source projects are decentralized by nature, but most organizations today have some element of decentralization
Challenges:– Quality control– Communication– Maintaining commitment
![Page 25: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/25.jpg)
Leading Practice
• Context• Questions and problems• Approaches• Tools
![Page 26: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/26.jpg)
Manual or automatic testing?
How do we ensure that the build complies with
the tests?
How does testing integrate with issue tracking?
Questions and problems
How do we enforce disciplined
testing?
How do we track 100s or 1000s of bugs
(including duplicates)?
How do we determine the cause
of test failure?
How do we make tests relevant?
Do we use unit, functional, integration, or acceptance testing?
Or all of them?
![Page 27: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/27.jpg)
Leading Practice
• Context• Questions and problems• Approaches• Tools
![Page 28: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/28.jpg)
Approaches
Test CoverageTest AutomationTest Driven DevelopmentBehavior Driven Development“Spec as Test” (the test/spec convergence)Exploratory TestingContinuous Integration
![Page 29: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/29.jpg)
Test coverage
The percentage of code or development that is tested.
In code, this might be measured by determining if every branch of code can result in a failing test condition (“Branch coverage”)
Wikipedia - http://en.wikipedia.org/wiki/Code_coverage
![Page 30: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/30.jpg)
Test automation
• Accepted as gospel modern dev communities– Regardless of how good they are at testing, accept
that they should automate as much as possible– Can lead to ignoring non-automatable testing
• In the SAP world we haven’t even accepted that full test coverage is desirable, much less automation
Test automation pushback -http://railspikes.com/2008/7/11/testing-is-overratedhttp://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-theo.html
Automated testing story on SDN (never completed) - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/4103
![Page 31: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/31.jpg)
A note on the following slides
I’m using bowling as an example because I’m copying from the internet. I don’t know much about bowling, so the examples are probably wrong. My assumption is that a game is usually 10 frames of two balls per frame. If the last frame is a spare or a strike, you get at most two additional frames.
Or something like that.
![Page 32: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/32.jpg)
Test Driven Development (TDD)
Design
• Technical specification
Design
• Write tests based on specs
Design
• Run tests
Test
![Page 33: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/33.jpg)
TDD Example
require ‘test/unit’require ‘test/unit/assertions’require ‘bowling’
class TC_Bowling < Test::Unit::TestCase
def setup bowling = Bowling.new end
def gutter_game 20.times { bowling.hit(0) } assert bowling.score == 0 endend
![Page 34: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/34.jpg)
Behavior Driven Development (BDD)
• Focus on the external behavior of the code• Abstract away implementation details• Additionally, BDD libraries tend to codify the
best practices of unit testing and TDD
![Page 35: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/35.jpg)
BDD example
require ‘spec’require ‘bowling’
describe Bowling do
it "should score 0 for gutter game" do
bowling = Bowling.new 20.times { bowling.hit(0) } bowling.score.should == 0
endend
![Page 36: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/36.jpg)
“Spec as Test” – or writing features
• Recently, the Ruby community has begun developing testing frameworks that are even closer to natural language.– Cucumber
• These frameworks wrap TDD and BDD frameworks and allow for “business-writable”, “business-readable”, executable test scripts.
![Page 37: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/37.jpg)
Feature exampleFeature (visible to the user)
Scenario: Gutter game Given I have bowled 20 balls and I have knocked over 0 pins per ball When I check the score Then I should have 0 points
Implementation (not visible to user)
require ‘spec/expectations’require ‘bowling’
Given /I have knocked over (\d+) pins per ball/ do |pins| @pins_per_ball = pinsend
Given /I have bowled (\d+) balls/ do |balls| @bowling = Bowling.new balls.times { @bowling.hit( @pins_per_ball ) }end
Then /I should have (\d+) points/ do |points| @bowling.score.should == pointsend
![Page 38: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/38.jpg)
Feature example cont.
Note that we can now implement many more tests with no more code/implementation:
Scenario: Perfect game Given I have bowled 12 balls and I have knocked over 10 pins per ball When I check the score Then I should have 300 points
Scenario: Bad game Given I have bowled 20 balls and I have knocked over 2 pins per ball When I check the score Then I should have 40 points
Scenario: Lots of spares
Given I have bowled 24 balls and I have knocked over 5 pins per ball When I check the score Then I should have ??? Points
Or maybe I need something a little different....
Scenario: Too long a game Given I have bowled 30 balls and I have knocked over 0 pins per ball Then I should receive an error
![Page 39: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/39.jpg)
Side-by-sideTDD BDD Spec as Test
require ‘test/unit’require ‘test/unit/assertions’require ‘bowling’
class TC_Bowling < Test::Unit::TestCase
def setup bowling = Bowling.new end
def gutter_game 20.times { bowling.hit(0) } assert bowling.score == 0 endend
require ‘spec/expectations’require ‘bowling’
describe Bowling do
it "should score 0 for gutter game" do
bowling = Bowling.new 20.times { bowling.hit(0) } bowling.score.should == 0
endend
Scenario: Gutter game
Given I have bowled 20 balls and I have knocked over 0 pins per ball
When I view the score
Then I should have 0 points
![Page 40: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/40.jpg)
TDD, BDD, and Test-as-Spec References
• Test Driven Design/Development– http://en.wikipedia.org/wiki/Test-driven_developmen
t– http://www.agiledata.org/essays/tdd.html
• Behavior Driven Development– http://behaviour-driven.org/
• Spec as test– Cucumber - http://cukes.info/– Rspec - http://rspec.info/ – http://www.pragprog.com/titles/achbd/the-rspec-bo
ok
![Page 41: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/41.jpg)
Exploratory Testing
• The practice of trying to break things– Career security for klutzes
• Exploratory testing appears informal, but can be structured and is a very important aspect of software testing.– Probably the most neglected form of testing in
open source projects
http://www.kohl.ca/blog/archives/000185.htmlhttp://www.kohl.ca/articles/ExploratoryTesting_MusicofInvestigation.pdf
![Page 42: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/42.jpg)
Continuous Integration
• The practice of automating not only your tests but your full commit-build-test cycle1. A commit new or changed code2. Triggers a full system build3. And an execution of the entire test suite
Cruisecontrol.rb -http://rubyforge.org/projects/cruisecontrolrbhttp://cruisecontrolrb.thoughtworks.com/
Hudson -https://hudson.dev.java.net/http://www.softwarebloat.com/2008/11/19/continuous-integration-blueprints-how-to-build-an-army-of-killer-robots-with-hudson-and-cucumber
![Page 43: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/43.jpg)
Leading Practice
• Context• Questions and problems• Approaches• Tools
![Page 44: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/44.jpg)
Tools
Source version control (svn, git)Testing libraries (rspec, cucumber, junit,
ABAPunit, jspec)Continuous Integration tools (cruisecontrol.rb,
Hudson)Issue tracking (Sourceforge, trac, Lighthouse,
Google Code)Agile methodologies (Scrum, XP, etc.)
![Page 45: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/45.jpg)
Modern methodology
• Modern quality assurance and testing– Leading practice in custom development, open
source, and agile projects• Test driven development (TDD), behavior driven
development (BDD), readable tests, executable requirements, continuous integration
– Overview of the SAP test automation technologies
– Road we're going to take through testing SAP
![Page 46: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/46.jpg)
The SAP World
• Manual• Unit testing– ABAP Unit– Java
• Functional testing– eCATT
![Page 47: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/47.jpg)
ABAP Unit
• Modeled on Java Unit• Excellent (if a bit unwieldy) for traditional unit
test automation or classes– Works well for TDD
• ABAP Objects (object-oriented ABAP)
• TDD Demo – ZBOWLING & ZBOWLING_TEST
http://help.sap.com/saphelp_nw70ehp1/helpdata/en/a2/8a1b602e858645b8aac1559b638ea4/frameset.htm
![Page 48: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/48.jpg)
Non-SAP
• We can use just about anything via RFCs• For Web Dynpros, BSPs, or Portal applications,
there are a lot of options using web-drivers that can automatically drive browsers
• Address in depth in future sessions
![Page 49: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/49.jpg)
Modern methodology
• Modern quality assurance and testing– Leading practice in custom development, open
source, and agile projects• Test driven development (TDD), behavior driven
development (BDD), readable tests, executable requirements, continuous integration
– Overview of the SAP test automation technologies– Road we're going to take through testing SAP
![Page 50: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/50.jpg)
The Road to the Future
• Developing techniques to support agile, open, decentralized development in an SAP landscape
• Using SAP tools and 3rd party tools• Shorter, more focused and hands-on sessions
![Page 51: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/51.jpg)
Start with this
http://en.wikipedia.org/wiki/File:V-model.JPG
![Page 52: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/52.jpg)
Get more parallel, less repetitive
http://en.wikipedia.org/wiki/File:V-model.JPG
Spec = Test
![Page 53: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/53.jpg)
Get faster
http://en.wikipedia.org/wiki/File:V-model.JPG
Test Automatio
n
![Page 54: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/54.jpg)
Get iterative
http://en.wikipedia.org/wiki/File:V-model.JPG
Continuous
Integration
![Page 55: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/55.jpg)
And end up here(or somewhere similar)
http://en.wikipedia.org/wiki/File:Scrum_process.svg
![Page 56: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/56.jpg)
Thanks
![Page 57: Testing Sap: Modern Methodology](https://reader037.vdocuments.mx/reader037/viewer/2022102704/54632822b1af9f92238b52ff/html5/thumbnails/57.jpg)
General references for inclusion• Open Collaboration within Corporations Using Software Forges - http://www.riehle.org/publications/2009/open-collaboration-within-corporations-using-software-forges/• Continuous integration tools
– http://rubyforge.org/projects/cruisecontrolrb - http://cruisecontrolrb.thoughtworks.com/ – https://hudson.dev.java.net/– http://www.softwarebloat.com/2008/11/19/continuous-integration-blueprints-how-to-build-an-army-of-killer-robots-with-hudson-and-cucumber
• Exploratory testing– http://www.kohl.ca/blog/archives/000185.html– http://www.kohl.ca/articles/ExploratoryTesting_MusicofInvestigation.pdf
• The ongoing revolution in software testing– http://www.kaner.com/pdfs/TheOngoingRevolution.pdf
• ABAP Unit– Spotlight on ABAP Unit Part 1 - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/1088
• Load testing– http://dannorth.net/2007/02/monkey-business-value
• Joel Spolsky – 12 steps to better code - http://www.joelonsoftware.com/articles/fog0000000043.html• Automated testing story on SDN - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/4103• JUnit - http://www.junit.org/ • Open Source
– http://www.paulgraham.com/opensource.html (What business can learn from open source)• Watir
– http://wtr.rubyforge.org/ • Spec as test
– Cucumber - http://cukes.info/– Rspec - http://rspec.info/ – http://www.pragprog.com/titles/achbd/the-rspec-book
• SAP testing– http://www.beteoblog.com/beteo-alm-miniguides/software-quality/– http://raa.ruby-lang.org/project/saprfc/ – Integration Tests in ABAP Development und Tool Support - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/7131 – XSLT - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/12173
• Test automation as panacea (not)– http://railspikes.com/2008/7/11/testing-is-overrated– http://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-theo.html
• BDD - http://behaviour-driven.org/• Blue Ruby - https://sap.na.pgiconnect.com/p16473929/ - http://www.slideshare.net/schmerdy/blue-ruby-sdn-webinar-1260181 -
https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/408a9a3b-03f9-2b10-b29c-f0a3374b19d8