the rise of chrome jonathan tamary and dror g. feitelson peerj computer science 1:e28, oct 2015
TRANSCRIPT
![Page 1: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/1.jpg)
The rise of Chrome
Jonathan Tamary and Dror G. FeitelsonPeerJ Computer Science 1:e28, Oct 2015
![Page 2: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/2.jpg)
The Question
• Since 2009 Chrome has come to dominate the desktop browsers market
• How has it done this?• Specifically, is it
technically better than the competition?
• Implication: is its rise different from that of Internet Explorer?
![Page 3: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/3.jpg)
“Technically Better”
• Better performance– Measure using common industry benchmarks– Add measurements of startup time
• Better conformance– Measure using common industry benchmarks
• Better features– Release features earlier– Focus on features that users consider important
![Page 4: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/4.jpg)
The Competition
• Microsoft Internet Explorer 8-11– Previous dominant browser– Bundled with Windows 98 , 76% in 1999– Antitrust case 1997-2002
• Mozilla Firefox 3-26– Branched from original Netscape code– Tried to compete with Explorer– Reached 30% in 2009
• Google Chrome 1-31
![Page 5: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/5.jpg)
PERFORMANCE
![Page 6: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/6.jpg)
Performance Benchmarks
Benchmark Content Response
SunSpider Javascript tasks Time
BrowserMark General browser performance
Score
CanvasMark <canvas> tag Score
PeaceKeeper Javascript tasks Score
Startup times Cold startup time Time
![Page 7: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/7.jpg)
Methodology
• Download and measure all browser versions– Some did not work with some benchmarks
• Perform measurements on a dedicated machine
• Use Win7 32-bit for versions till may 2011, 64-bit for later versions– Results on the two systems were consistent
• Repeat measurements and calculate standard error
![Page 8: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/8.jpg)
Data Collection
Benchmark Rep Missing
Explorer Firefox Chrome
SunSpider 3 30, 31
BrowserMark 3 8 3, 3.5, 3.6 1
CanvasMark 3 8 3 1, 2, 3
PeaceKeeper 3 1, 2, 3, 4
Startup times 20
![Page 9: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/9.jpg)
SunSpider• Developed by WebKit• Measure core Javascript
performance: tasks rather than microbenchmarks
• Debate whether representative
• Explorer improvement attributed to dead-code elimination, perhaps specifically to improve SunSpider performance
![Page 10: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/10.jpg)
BrowserMark 2.0
• Developed by Rightware, originally for mobile and embedded
• Multiple aspects of general browser perofrmance– Page load/resize– WebGL– HTML5, CSS3– Canvas
![Page 11: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/11.jpg)
CanvasMark 2013
• Test HTML5 <canvas> tag: container for graphics drawn with Javascript
• Stress test using– Bitmap operations– Alpha blending– Shadows– Text
• Goal: stay > 30fps
![Page 12: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/12.jpg)
PeaceKeeper
• Developed by FutureMark
• Focus on Javascript use– <canvas>– DOM tree ops– Parsing– Video formats– multithreading
![Page 13: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/13.jpg)
Startup Times
• At boot run script to launch browser 1 min later
• Take timestamp just before launching and pass to browser via URL parameter
• Browser loads page that takes another timestamp and sends to server
• Difference between timestamps is startup time
![Page 14: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/14.jpg)
CONFORMANCE
![Page 15: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/15.jpg)
Conformance Benchmarks
Benchmark Content Response
HTML5 compliance
HTML standard Score
CSS3 test CSS standard Score
Browserscope security
Security enhancing features
Tests passed
![Page 16: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/16.jpg)
HTML5 Compliance
• The language to describe web pages
• HTML5 introduced in 2008 and approved 2014
• Benchmark checks supported features
![Page 17: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/17.jpg)
CSS3 Test
• The language to describe web page style
• Check recognized elements of CSS spec
• Does not check wuality of implementation
![Page 18: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/18.jpg)
Browserscope Security
• Community-driven project to profile prowsers
• Checks support for Javascript APIs for safe interactions
• Score is tests passed
![Page 19: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/19.jpg)
Other Browsers
• We compared the 3 top browsers• There are others– Opera– Safari
• Why did Chrome gain market share while they did not?
• If they are just as good technically, then Chrome’s advantage is only in marketing
![Page 20: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/20.jpg)
Opera
• Main contender in Windows desktop market
• Opera is technically inferior to Chrome
![Page 21: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/21.jpg)
FEATURES
![Page 22: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/22.jpg)
Feature Selection
• Start with 43 features of modern browsers• Remove 11 that were included in Chrome 1
and already existed in Firefox and Explorer– Tabs, history manager, pop-up blocking, …
• Remove 7 that were introduced at about the same time by all 3 browsers– Private browsing, full screen, …
• Use 25 remaining features
![Page 23: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/23.jpg)
Add-ons manager Multiple usersDownload manager AppsAuto-updater Developer toolsCaret navigation Personalized new tabPinned sites Click-to-playSync Print previewSession restore (automatically) Per-site security configurationCrash/security protection Web translationMalware protection Spell checkingOutdated plugin detection Built-in PDF viewerDo not track SandboxingThemes RSS readerExperimental features
![Page 24: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/24.jpg)
Competitive Advantage
• Feature should be released ahead of competition by a meaningful margin
• Our definition: more than one release cycle
• Gives advantage to browsers with slow release (Internet Explorer)
![Page 25: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/25.jpg)
Wins and Losses
• A browser wins if it releases a feature ahead of all the competition
• A browser loses if it releases a feature behind all the competition
• Each feature can have at most one winner and one loser
• 7 features with no winner or loser were removed
![Page 26: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/26.jpg)
Wins and Losses
Wins Losses Chrome 6 5Firefox 5 6Internet Explorer - 13
![Page 27: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/27.jpg)
Feature Importance Survey
• Online survey• 254 participants– HUJI CS facebook page– TAU CS facebook page– Reddit.com/r/SampleSize
• Rank each of 25 features on a 5-point scale1 = least important5 = most importantUse a relative scale (compare features to each other)
![Page 28: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/28.jpg)
Analysis Questions
• Are features that Chrome won more important than those that Firefox won?
• Are features that Chrome lost less important than those that the other two browsers lost?
• Are features that Chrome won more important than those it lost?
• Need to compare scores for sets of features
![Page 29: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/29.jpg)
Statistical Analysis
Conventional approach:• Find average importance grade of features in
each set• Use statistical test for significance
This is wrong!• Scores do not come from a valid interval scale
12 is not necessarily the same as 34• Averaging is meaningless
![Page 30: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/30.jpg)
Statistical Analysis
• Use methodology developed to compare brands
• Brand A is superior to B if distribution of opinions about A dominates distribution of opinions about B in the stochastic order sense
• In plain English: the distribution is skewed toward higher scores (the CDF is lower)
• In our case, “brands” are sets of features(not Microsoft, Google, Mozilla)
![Page 31: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/31.jpg)
Statistical Analysis
• The problem: neither distribution may dominate the other (the CDFs cross)
• Solution procedure:1. Identify homogeneous brands with clustering.
These are brands that cannot be distinguished.2. Find widest collapsed scale. Collapsing unites
adjacent score levels to achieve dominance, but we want to keep as many levels as possible.
3. Verify that resulting dominance is significant.• Due to Yakir & Gilula, 1998
![Page 32: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/32.jpg)
Results for WinsRank Browser Wins Importance scores distribution
1 2 3 4 51 Chrome 6 0.16 0.20 0.24 0.23 0.172 Firefox 5 0.27 0.22 0.23 0.20 0.093 Internet
Explorer0 -- -- -- -- --
![Page 33: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/33.jpg)
Results for LossesRank Browser Losses Importance scores distribution
1 2 3-4 51 Firefox 6 0.17 0.16 0.45 0.22
Internet Explorer
13
2 Chrome 5 0.18 0.16 0.44 0.22
![Page 34: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/34.jpg)
Results for ChromeRank Class Features Importance scores distribution
1-2 3 4 51 Losses 5 0.33 0.18 0.27 0.22
2 Wins 6 0.36 0.24 0.23 0.17
![Page 35: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/35.jpg)
END GAME
![Page 36: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/36.jpg)
Summary of ResultsBenchmark ResultSunSpider Chrome was best through 2010, now
Internet Explorer is significantly better
BrowserMark 2.0 Chrome is best, Explorer worstCanvasMark 2013 Chrome is relatively good but inconsistent,
Firefox worst
PeaceKeeper Chrome is significantly betterStart-up times initially Chrome was better but now Firefox
is better, Explorer has deteriorated
HTML5 Compliance Chrome is better, Explorer worstCSS3 Test Chrome is betterBrowserscope Security
Chrome is better, Firefox worst
![Page 37: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/37.jpg)
Summary of Results
• Crome won on 6 features and lost on 5• Firefox won on 5 features and lost on 6• Internet Explorer lost on 13 features• Chrome’s wins were more important than
Firefox’s wins• The losses of all browsers were equally
important• Chrome’s losses were slightly more important
than its wins
![Page 38: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/38.jpg)
Implications
• Internet explorer is proprietary• Firefox is open source– More innovative and better than product from
leading software firm• Chrome is related to open-source Chromium– But how “open” is it?
• Main factor apparently company/organization and not development style
• Firefox & Chrome also moved to rapid releases– Slow releases could contribute to Explorer’s demise
![Page 39: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/39.jpg)
Threats to Validity
• How to measure market share– netmarketshare.com claims Explorer dominates
• Focus on technical aspects– Ignores marketing campaign and Google brand
• Didn’t check all smaller browsers– If better than Chrome then marketing was decisive
• Used benchmarks do not cover all aspects (and not clear exactly what they do)– But writing new ones suffers no lesser threats
![Page 40: The rise of Chrome Jonathan Tamary and Dror G. Feitelson PeerJ Computer Science 1:e28, Oct 2015](https://reader035.vdocuments.mx/reader035/viewer/2022062315/5697bf8e1a28abf838c8ccb6/html5/thumbnails/40.jpg)
Conclusions
• Chrome’s rise is consistent with technical superiority
• But it is not necessarily the result of technical superiority
• The Google brand name and the marketing campaign may have had significant effect