browserbite: accurate cross-browser testing via machine learning over image features nataliia...

Browserbite: Accurate Cross-Browser

Testing via Machine Learning Over

Image Features Nataliia Semenenko*, Tõnis Saar** and

Marlon Dumas*

*{nataliia,marlon.dumas}@ut.ee, Institute of Computer Science,

University of Tartu, Estonia**tonis.saar@stacc.ee,

Browsrbite and STACC, Tallinn, Estonia

Outline

• Introduction• Visual cross-browser testing• Machine learning model• Results and future work

Cross-browser visual testing

Internet Explorer 9 Internet Explorer 8

Where’s that button?

Goal

• Develop method for cross-browser visual layout testing

• Replace human labor in visual testing• Evaluate detected errors

Methods• DOM (Document Object Model) based:

Mogotest (www.mogotest.com), Browsera (www.browsera.com)

• Image processing – non-invasive black box testing – Our current approach

Web page Static image

Cross-Browser Visual testing

Web page visual segmentation

• Image segmentation into regions of interest (ROI)

• ROI comparison

www.htcomp.ee

ROI comparison

• Position• Size• Geometry• Correlation

ROI from WIN7 Chrome

ROI from WIN7 IE8

VS

Visual testing results

• Test set of 140 web pages from alexa.com• 98% recall• 66% precision

Web page Static imageImage

segmentation(into ROIs)

ROI comparison

Example of true positive Example of false positive

ROI comparison + ML

Web page Static imageImage

segmentation(into ROIs)

ROI comparison

Classification

Machine learning

• 140 most popular websites of Estonia according to www.alexa.com

• 1200 potential incompatibilities • 40 subjects from 6 countries• Two classes :False positive vs True postive• Each ROI pair had 8 judgments• Inter-rater reliability 0,94

ROI features

• 10 histogram bins• Correlation index• Horizontal and vertical position• Horizontal and vertical size• Configuration index• Mismatch Density

Machine learning

• Neural network• Three layers• 11 neurons in hidden layer• Five-fold cross-validation

• Classification tree

Results and Conclusions

Measure Plain Browserbite

Mogotest Classification tree

Neural network

Precision 0.66 0.75 0.844 0.964

Recall 0.98 0.82 0.792 0.886

F-score 0.79 0.78 0.81 0.923

Results and conclusions

1. Choudhary, S.R., Prasad, M.R., and Orso, A. (2012). CrossCheck: Combining Crawling and Differencing to Better Detect Cross-browser Incompatibilities in Web Applications. (ICST), 2012 IEEE Fifth International Conference On, pp. 171–180.

2. Choudhary, S.R., Versee, H., and Orso, A. (2010). WEBDIFF: Automated identification of cross-browser issues in web applications. (ICSM), pp. 1–10.

Tool Mogotest CrossCheck [1] WebDiff [2] BB+ML

Precision 75% 36% 21% 96%

Future work

• Combination of image processing and DOM methods

• Dynamic content suppression

Thank You!