S A U C E L A B S T E S T C O V E R A G E R E P O R T A U G U S T 2 0 1 6

Customers often ask us what browser and OS combinations they

should test against. There really is no single right answer as it depends

on your app and your users. However, with over 640 million tests

completed to date, we have a lot of data that can help you in forming

a test coverage strategy.

This coverage report summarizes anonymous data from tests run on

Sauce Labs in the last 90 days. It is useful for identifying testing trends

and understanding what browsers / OS combinations might best apply

to your organization.

3 Top OSes

3 Top Browsers

4 Unique Number of OSes

4 Mac, Windows and Linux by Industry

5 Top Browsers by Industry

5 Chrome Browser Versions

6 FireFox Browser Versions

6 Other Browsers

6 Summary

TABLE OF CONTENTS

TOP OSes

This is the percentage of tests performed by OS. Windows continues to be the

top platform tested, followed by Mac OS.

TOP OSes USED ON SAUCE L ABS (BY TEST VOLUME)

8.3%

19.4%

72.3%

Windows

Mac OS

Linux

TOP BROWSER

This is the percentage of testing performed on browsers (regardless of

platform). Chrome has recently topped Internet Explorer as the most used

browser world wide, so it’s no surprise that this is the most tested browser on

Sauce. Firefox is #2 followed by Internet Explorer. The Edge browser is very

small now, but expected to grow as Windows 10 becomes more mainstream.

According to Mozilla, 57% of browser use is Chrome. Many companies dictate

which browsers employees can use and many consumers use the default

browser that comes with their system. The following table shows general

browser preferences by region.

TOP BROWSERS USED ON SAUCE L ABS (BY TEST VOLUME)

6.7%

10%

45%

37.8%

Google Chrome

Firefox

Internet Explorer

Safari

Learn more at saucelabs.com

3

Country Chrome Firefox Internet Explorer

USA 46% 12% 19%

Germany 33% 34% 12%

France 47% 21% 11%

Japan 38% 15% 27%

Indonesia 49% 36% 2%

Source: Mozilla - http://tinyurl.com/h7ywx3n

UNIQUE NUMBER OF OSES

This is a measure of how many different OSes are used by industry.

The more platforms an app is tested against, the smaller the chance

your users will experience poor quality. Perhaps it’s not a big surprise

that the software industry tests across the widest variety of browser / OS

combinations. Rounding out the top 3 is Finance and Media where the stakes

for quality are high. It might surprise you to learn the average number of

browsers that our customers test against is 114.

UNIQUE OSes TESTED (BY INDUSTRY )

Technolo

gy

Finan

ce

Media

/Adve

rtisin

g

Retail

Non-P

rofit

E-Com

merc

e

Consulti

ng

Consum

er Pro

ducts

Tele

com /

Comm

Serv

ices

Educatio

n

Hosp

itals

& Heal

thcar

e

Ave

rag

e

240

180

120

60

0

201

175.5

156143.5 143

107 102

86

59

42.5 42.5

MAC, WINDOWS AND LINUX BY INDUSTRY

This is a measure of the mix of OSes tested by industry. E-commerce tends to

test heavily against Macs, for all other industries, Windows dominates.

Windows Mac Linux

OS TESTS BY INDUSTRY

Technolo

gy

Retail

Consulti

ng

Finan

ce

Tele

com /

Comm

Serv

ices

E-Com

merc

e

Educatio

n

Consum

er Pro

ducts

Hosp

itals

& Heal

thcar

e

Media

/Adve

rtisin

g

Non-P

rofit

100.00%

75.00%

50.00%

25.00%

0.00%

Industry

Learn more at saucelabs.com

4

TOP BROWSERS BY INDUSTRY

This is a measure of total number of browsers tested by industry. Firefox and

Chrome dominate across most industries. In media and non-profit FireFox

is the leading browser. Again, in ecommerce we see significant use of Safari

where it is assumed that Macs are a significant factor.

Chrome Firefox IE Safari

BROWSER TESTS BY INDUSTRY

Technolo

gy

Retail

Consulti

ng

Finan

ce

Tele

com /

Comm

Serv

ices

E-Com

merc

e

Educatio

n

Consum

er Pro

ducts

Hosp

itals

& Heal

thcar

e

Media

/Adve

rtisin

g

Non-P

rofit

100.00%

75.00%

50.00%

25.00%

0.00%

Industry

CHROME BROWSER VERSIONS

Chrome is an evergreen browser which means it updates automatically. So it is

expected that testing follows a regular upgrade cadence and we generally see

testers upgrading to the latest versions. This was true in late 2015, however,

in early 2016 version 48 dominated for half the year.

L ATEST GOOGLE CHROME VERSION ADOPTION ( V. 44 - 51 )

10/1/15 1/1/16 4/1/16 7/1/16

5150494847464544

Learn more at saucelabs.com

5

FIREFOX BROWSER VERSIONS

FireFox finds its heaviest use outside North America. With regular updates, we

find that users tend to test on the latest versions to ensure their apps continue

to function properly.

L ATEST F IREFOX VERSION ADOPTION ( V. 40 - 47)

4746454443424140

10/1/15 1/1/16 4/1/16 7/1/16

OTHER BROWSERS

For IE and Safari, people generally upgrade upon new OS versions so

organizations generally test on the major versions. As a result, we don’t see

much variation in testing and have not included them in this report.

SUMMARY

As you can see, the browser and OS combinations that matter to your

organization will differ based on your industry and the demographics of your

users. Regardless, the more test coverage you have, means a higher quality

app and fewer bugs that your users will experience. For a complete list of the

700+ browser and OS combinations provided by Sauce Labs, please go to

www.saucelabs.com/platforms.

Learn more at saucelabs.com

6

Sauce Labs provides the world’s largest cloud-based platform for the

automated testing of web and mobile applications. Its award-winning

service eliminates the time and expense of maintaining an in-house testing

infrastructure, freeing development teams of any size to innovate and

release better software, faster.

Sauce Labs is a privately held company funded by Toba Capital, Salesforce

Ventures, Triage Ventures and the Contrarian Group. For more information,

please visit saucelabs.com.

SAUCE LABS INC. 539 BRYANT STREET #303 SAN FRANCISCO, CA 94107 USA

ABOUT SAUCE L ABS