improving software quality and devop automation with stamp, ow2con'16, paris
TRANSCRIPT
![Page 1: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/1.jpg)
STAMPSoftware testing amplification to increase quality
in DevOps
1
![Page 2: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/2.jpg)
2
DevOps
![Page 3: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/3.jpg)
3
DevOps
Test assets are available
BUT
They are manually developed
and their quality is not sufficient
![Page 4: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/4.jpg)
4
DevOps
The main objective of STAMP is to
automatically transform existing
test assets in order to detect
regression bugs before
production and drive down the
cost of software testing.
![Page 5: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/5.jpg)
5
STAMP in DevOps
![Page 6: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/6.jpg)
Continuous test amplification
6
![Page 7: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/7.jpg)
Continuous test amplification
7
behav. diff.
![Page 8: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/8.jpg)
Continuous test amplification
8
behav. diff.
![Page 9: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/9.jpg)
Continuous test amplification
9
behav. diff. perf. diff.
![Page 10: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/10.jpg)
Continuous test amplification
10
behav. diff. perf. diff.
![Page 11: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/11.jpg)
Continuous test amplification
11
behav. diff. perf. diff.
![Page 12: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/12.jpg)
Unit test cases amplification
12
Program
P
Input space Output space
state
![Page 13: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/13.jpg)
Unit test cases amplification
13
Program
P
Input space Output space
state
![Page 14: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/14.jpg)
DSpot: amplification of JUnit test suites
14
original test suite and
program under test
update in the program
under test
PDSpot
TS
P'
TS++
![Page 15: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/15.jpg)
DSpot
•First experiments with amplification
•Automatic transformation of Junit test cases
15
P
TS
P'
TS+search-based input space amplification
observation space
amplification
run and look for differences
TS++
![Page 16: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/16.jpg)
Search-based input space amplification
• Mutate existing test cases
• transform literal values
• replace literal values by others found in the scope
• add new invocations to methods not well covered
• create new objects
• Search-based
• ‘stack’ transformations iteratively
• select the ones that have a good fitness (e.g. cover new branches)
16
![Page 17: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/17.jpg)
Observation space amplification
• Remove all assertions from original test cases
• keep invocations that can be inside the assertion
• Add new observation points
• run all test cases on original program
• record all values that can be observed (getters, is_*, etc.)
• Filter new observation points
• several runs of all test cases with observation points on the original program
• discard all points that vary (e.g., OS-dependent values, IO, etc.)
17
![Page 18: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/18.jpg)
Example
18
public void testSearch() {
final ArrayStack<E> stack = makeObject();
stack.push(((E)("First Item")));
stack.push(((E)("Second Item")));
assertEquals("Top item is \'Second Item\'", 1, stack.search("Second Item"));
assertEquals("Next Item is \'First Item\'", 2, stack.search("First Item"));
assertEquals("Cannot find \'Missing Item\'", -1, stack.search("Missing Item"));}
public void testSearch_cf5() {
final ArrayStack<E> stack = makeObject();
stack.push(((E)("First Item")));
stack.push(((E)("Second Item")));
E o_5 = stack.peek();
assertFalse(((java.lang.String)o_5).isEmpty());
E o_6 = stack.pop();
assertFalse(((java.lang.String)o_6).isEmpty());
Object o_7_0 = stack.search("Second Item");
Object o_9_0 = stack.search("First Item");
assertEquals("Cannot find \'Missing Item\'", -1, stack.search("Missing Item"));
assertEquals(((java.lang.String)o_5).length(), 11);
assertEquals(((java.lang.String)o_6).length(), 11);}
new calls and asserts
extracted from asserts
new asserts
![Page 19: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/19.jpg)
Methodology
19
Unit test ampli
Config. test ampli
Online test ampli
![Page 20: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/20.jpg)
Methodology
20
Micro service
architecture
Unit test ampli
Config. test ampli
Online test ampli
![Page 21: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/21.jpg)
Methodology
21
Micro service
architecture
Unit test ampli
Config. test ampli
Online test ampli
Information
managementSoftware
quality
Cloud
computing
E-Health
Smart cities
![Page 22: Improving software quality and devop automation with STAMP, OW2con'16, Paris](https://reader031.vdocuments.mx/reader031/viewer/2022030311/58ee74a81a28abf0678b461b/html5/thumbnails/22.jpg)
STAMP
•3-year project
•9 EU partners
•Full OSS
•Starts 01/12/16
22