evolutions in browser security

20
   !"!#$%& ()*+, !"#$%&'#() '( +,#-)., /.0%,'&1  23!45/ 64 +378/!3 /!9:362; <!3=73>?49! ?%&@#, A 3B(C1 ?D,BE) "#$%#&$' ()&* +,+-.*/ 0%&$1 +22%$2+/$* %$*3-/* 1%45 677 8+0* /$*/* 94,:39/$: 0$/'$$, ;<<= +,: ;<>? &, + 945@+%&*4, 41 @)&*)&,2 +,: *49&+--. $,2&,$$%$: 5+-'+%$ A7BCD @%4/$9/&4, 0. /)$ -$+:&,2 0%4'*$%*E F&23%$ > %$#$+-* /%$,:* &, @%4/$9/&4, -$#$-* 41 /)$ 143% -$+:&,2 0%4'*$%*G 945@+%&,2 9450&,$: /$*/ %$*3-/* 1%45 ;<<= /4 /)$ %$9$,/ ;<>; +,: ;<>? *94%$*E ='F%,. G A H.BC'(F +,#-)., >B$-B,. B(C <@')@'(F +$#0I 3B&.) JKLLMN KLGKN KLGOP ;<<= ;<>; ;<>? ;<<= ;<>? ;<<= ;<>; ;<>? ;<<= ;<>;    H    #    $    %    +    2    $    I     )    &    *     ) H#$%+2$ C+-'+%$ <J ><J ;<J ?<J K<J L<J M<J N<J O<J =<J ><<J <J ;<J K<J M<J O<J ><<J    >    B     $    -    B    ,    . <@')@'(F P)%45$ F&%$14Q RB 7+1+%& H#$%+2$ I)&*) H#$%+2$ C+-'+%$

Upload: yuventia-marsel-mamesah

Post on 02-Nov-2015

226 views

Category:

Documents


0 download

DESCRIPTION

Evolutions in Browser Security

TRANSCRIPT

  • ANALYST BRIEF

    Evolutions in Browser Security TRENDS IN BROWSER SECURITY PERFORMANCE

    Author Randy Abrams

    Overview This analyst brief aggregates results from NSS Labs tests conducted between 2009 and 2013 in a comparison of phishing and socially engineered malware (SEM) protection by the leading browsers. Figure 1 reveals trends in protection levels of the four leading browsers, comparing combined test results from 2009 to the recent 2012 and 2013 scores.

    Figure 1 Leading Browser Malware and Phishing Block Rates (2009, 2012, 2013)

    2009

    2012 2013

    2009

    2013

    2009

    2012

    2013

    2009

    2012

    Average Ph

    ish

    Average Malware

    0%

    10%

    20%

    30%

    40%

    50%

    60%

    70%

    80%

    90%

    100%

    0% 20% 40% 60% 80% 100%

    Malware

    Phishing

    Chrome

    Firefox

    IE

    Safari

    Average Phish

    Average Malware

  • NSS Labs Analyst Brief Evolutions In Browser Security

    2

    In the NSS tests, the browsers are rated on performance in four categories:

    Average phishing block rate Zero-hour phishing block rate Average SEM block rate Zero-hour SEM block rate

    Internet Explorer (IE) shows a consistently superior ability to block SEM, while providing competitive phishing protection; it leads the tested browsers in combined protections for these categories. Googles Download Protection technology has improved significantly over time, placing it behind IE but well ahead of Firefox and Safari. Both Firefox and Safari lead the other browsers in phishing protection but provide negligible protection against SEM attacks.

    This analyst brief includes data from previously published NSS phishing tests conducted in 2009, 2012, and 2013, as well as SEM tests published every year from 2009 to 2013. In 2010, two SEM tests were published.

    Figure 2 shows the overall performance of the browsers for the 2013 browser phishing and SEM tests.

    Figure 2 2013 Combined Phishing And Malware Block Rates

    Figure 2 treats all protection metrics in 2013 equally. Later in this analyst brief, Figure 15 will add weighting based on the relative protection importance of the tested parameters in order to provide a more realistic ranking of the browsers in 2013. Figure 14 provides an aggregation of all of the tests from 2009 to 2013, with emphasis placed on freshness and relative importance of the test metrics.

    52%

    53%

    76%

    89%

    0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

    Firefox

    Safari

    Chrome

    IE

  • NSS Labs Analyst Brief Evolutions In Browser Security

    3

    NSS Labs Findings The browser is the first line of defense against multiple web-based threats; however, with a maximum

    historical protection rate of just 80 percent, the browser should not be the only line of defense. Products that do not provide the bulk of their protection in the earliest hours of an attack are not meeting the

    security requirements of todays threatscape. Microsofts Internet Explorer continues to provide the best combination of malware and phishing protection. The application reputation technologies used by browsers from both Microsoft and Google provide a

    significantly safer browsing experience than do the browsers from Apple and Mozilla. User education is often better protection against social engineering attacks than browser technologies.

    NSS Labs Recommendations Invest in awareness education about social engineering for all users. Evaluate trends that may indicate the need for browser replacement. Select and use security products that augment the protective capabilities of the browser.

  • NSS Labs Analyst Brief Evolutions In Browser Security

    4

    Table of Contents

    Overview ................................................................................................................................ 1

    NSS Labs Findings .................................................................................................................... 3

    NSS Labs Recommendations ................................................................................................... 3

    Analysis .................................................................................................................................. 6 Phishing Trends and Threats ........................................................................................................................ 6 NSS Empirical Results: Phishing Protection .................................................................................................. 7 NSS Empirical Results: Socially Engineered Malware Protection ................................................................. 9 Combined Protection Effectiveness ........................................................................................................... 11 Aggregate Values .................................................................................................................................... 11 Weight A Minute ..................................................................................................................................... 11 Evaluating the Data ................................................................................................................................ 11

    The Great Equalizer .................................................................................................................................... 14

    Appendix A: Raw Data .......................................................................................................... 15

    Appendix B: Weighting Formulas .......................................................................................... 17 Protection Multipliers: ............................................................................................................................... 17 Year Multipliers: ......................................................................................................................................... 17

    Reading List .......................................................................................................................... 19

    Contact Information .............................................................................................................. 20

  • NSS Labs Analyst Brief Evolutions In Browser Security

    5

    Table of Figures Figure 1 Leading Browser Malware and Phishing Block Rates (2009, 2012, 2013) .................................................... 1

    Figure 2 2013 Combined Phishing And Malware Block Rates .................................................................................... 2

    Figure 3 Unique Phishing Attacks .............................................................................................................................. 6

    Figure 4 APWG Phishing Uptime Statistics ................................................................................................................ 7

    Figure 5 Mean Block Rate for Phishing ...................................................................................................................... 7

    Figure 6 Zero-Hour Phishing Block Rate ..................................................................................................................... 8

    Figure 7 Time to Block Phishing Attacks Relative to Uptime Trends (Hours) ............................................................. 9

    Figure 8 Mean Block Rate for Socially Engineered Malware ..................................................................................... 9

    Figure 9 Zero-Hour Socially Engineered Malware Block Rate .................................................................................. 10

    Figure 10 Content Agnostic Malware Protection Breakout ..................................................................................... 10

    Figure 11 Combined Test Results (Not Weighted) ................................................................................................... 12

    Figure 12 Time Weighted ......................................................................................................................................... 12

    Figure 13 Protection Weighted ................................................................................................................................ 13

    Figure 14 Time and Protection Weighted Scores ..................................................................................................... 13

    Figure 15 2013 Weighted Scores ............................................................................................................................. 13

    Figure 16 Mean Block Rate for Phishing .................................................................................................................. 15

    Figure 17 Zero-Hour Block Rate ............................................................................................................................... 15

    Figure 18 Mean Block Rate for SEM ......................................................................................................................... 15

    Figure 19 Zero-Hour Block Rate for SEM ................................................................................................................. 16

    Figure 20 Time Weighting ........................................................................................................................................ 17

    Figure 21 Protection Type Weighting ...................................................................................................................... 17

    Figure 22 Type and Year Weighting ......................................................................................................................... 17

  • NSS Labs Analyst Brief Evolutions In Browser Security

    6

    Analysis Socially engineered malware and phishing attacks are two of the most significant threats against which web browsers must defend. NSS has for several years tested the leading browsers for their ability to protect against these attacks; however, each test has been presented as a stand-alone snapshot in time. While these real-world snapshot tests yield useful information, a correlated report is equally valuable in order to assess trends and establish vendor track records. This analyst brief examines the historical performance of browsers against phishing and against socially engineered malware attacks. The browsers are evaluated against each other and against the phishing threatscape. If the best performing product affords little protection, then the worst performing product is not significantly different. Fortunately, there are browsers that are addressing the challenges and that are able to provide significant protection for users.

    Phishing Trends and Threats The Anti-Phishing Working Group (APWG) has collected and published statistics about phishing attacks for several years. The APWG Phishing Attack Trends1 and Global Phishing Survey2 reports provide important insight into the phishing problem.

    From Figure 3 it can be inferred that although the number of unique phishing emails and web sites has varied from 2009 to 2012, the scope of the problem remains significant. The uptick in unique phishing sites discovered in 2012 is not accompanied by a significant uptick in reports of unique phishing emails. While this may be due to under-reporting of phishing email, it is likely indicative of cyber criminals increasingly using redirects in an attempt to compensate for the declining lifetimes of their attacks.

    Figure 3 Unique Phishing Attacks

    1 http://www.apwg.org/resources/apwg-reports/ 2 http://www.apwg.org/resources/apwg-reports/whitepapers

    0

    100,000

    200,000

    300,000

    400,000

    500,000

    600,000

    700,000

    2009 2010 2011 2012 Unique Phishing Emails Reported Unique Phishing Web Sites Discovered

  • NSS Labs Analyst Brief Evolutions In Browser Security

    7

    One of the critical metrics surrounding a browsers effectiveness in combatting phishing attacks is how quickly it adds protection once an attack is live. Figure 4 illustrates the general decline in the lifetimes of phishing sites. In 2012, the average phishing site was live for just under 25 hours, and the median lifetime was approximately 12 hours. Products that do not provide the bulk of their protection in the earliest hours of an attack are not meeting the security requirements of todays threatscape.

    Figure 4 APWG Phishing Uptime Statistics

    NSS Empirical Results: Phishing Protection NSS tested the leading browsers for phishing protection in 2009, 2012, and 2013, with the results presented in Figure 5 and Figure 6.

    Figure 5 Mean Block Rate for Phishing

    In 2009, only IE and Firefox provided competitive block rates for phishing, with results in 2012 narrowing the differences to a four-point spread between browsers. In 2013, Firefox and Safari posted modest improvements in their scores, while Chrome dropped by 2 percent. IE has a trend of solid performance, but the browsers phishing block rate declined from 92 percent in the 2012 Browser Security Comparative Analysis Report (CAR) on Phishing Protection to 83 percent in the 2013 CAR on Phishing Protection.

    0

    20

    40

    60

    80

    1H2009 2H2009 1h2010 2h2010 1h2011 2h2011 1h2012 2h2012 Average Uphme (Hours) Median Uphme (Hours)

    0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

    100%

    2009 2012 2013

    Chrome

    Firefox

    IE

    Safari

    Average

  • NSS Labs Analyst Brief Evolutions In Browser Security

    8

    The lower results in the 2013 test should be evaluated against future tests to determine if the decline in IEs mean block rate in the 2013 test indicates a problematic trend.

    The time required to add new phishing sites is an important metric when determining the relevance of the mean block rate to consumer protection. For example, a browser that blocks more phishing sites in the first 12 hours will provide better protection than a less responsive browser that achieves a better block rate in the long run. Historically, NSS testing has found that the browsers with the best early detection continue to lead until the end of the test; however, this may not always be the case.

    Figure 6 Zero-Hour Phishing Block Rate

    During the 2013 test, all browsers showed improvement over their historical zero-hour block rates, as depicted in Figure 6. A metric that has a high correlation to the zero-hour block rate is the average time required to add protection for new phishing sites.

    Figure 7 shows the APWG average phishing site uptime statistics and the mean phishing site uptime statistics overlaid with the results of the tested browser performance for average time to add protection for new phishing sites. (The APWG statistics for the first half of 2013 were not available at the time of writing.) Note that the browser phishing protection tests were performed only in 2009, 2012, and 2013, so performance in 2010 and 2012 is graphed in a linear fashion and may not reflect actual performance in those two years. Safari is the only browser to have had a worse response time to phishing attacks than either the mean or the average uptime for phishing sites since NSS began testing browsers. But Apple has dramatically improved its performance, and Safari posted the fastest response times in the most recent test report.

    The median uptime for phishing attacks is significantly lower than the average response time and is the more important metric. All of the browsers are adding protection very quickly, with IE requiring 2.6 hours and Safari averaging 30 minutes. Firefox, however, has demonstrated the most consistent protection for phishing over time.

    0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

    100%

    2009 2012 2013

    Chrome

    Firefox

    IE

    Safari

    Average

  • NSS Labs Analyst Brief Evolutions In Browser Security

    9

    Figure 7 Time to Block Phishing Attacks Relative to Uptime Trends (Hours)

    NSS Empirical Results: Socially Engineered Malware Protection Socially engineered malware (SEM) refers to an attack that deceives users into downloading and installing malicious software. In recent years, rogue antivirus programs have been at the forefront of SEM; however, there are many types of malicious programs that criminals use in conjunction with social engineering for financial gain. Figure 8 demonstrates that there are, and historically have been, dramatic differences in browser protection against SEM.

    Figure 8 Mean Block Rate for Socially Engineered Malware

    NSS conducted six browser malware protection tests between 2009 and 2013, and IE significantly outperformed the competition in all six texts. Only recently has Chrome become a viable option that provides significant malware protection for users. While Chrome, Firefox, and Safari all use Googles Safe Browsing API, Chrome alone incorporates Googles Content Agnostic Malware Protection technology (CAMP). Prior to 2012, all three of the browsers using Googles Safe Browsing API performed comparably.

    0

    10

    20

    30

    40

    50

    60

    70

    2009 2010 2011 2012 2013

    Chrome

    Firefox

    IE

    Safari

    APWG Average Uphme

    APWG Median Uphme

    u Google Buys VirusTotal

    0%

    20%

    40%

    60%

    80%

    100%

    2009 Q1 2010 Q3 2010 2011 2012 2013

    Chrome Firefox IE Safari

  • NSS Labs Analyst Brief Evolutions In Browser Security

    10

    When Google acquired VirusTotal, it was widely assumed that the service would be used to improve Chromes malware blocking abilities. The 2013 results do show an improvement in Chromes mean block rate, but also a significant drop in zero-hour protection. If Google continues to improve Chromes SEM protection, it may be difficult to differentiate the contribution of the VirusTotal acquisition from ongoing investment in Googles Download Protection technology.

    As with phishing attacks, response time is critical when providing protection against malware attacks. Figure 9 shows browser performance graphed at zero hour against SEM attacks. IE significantly outperforms the competition in all six tests. Both Firefox and Safari have declined in SEM protection since 2010.

    Figure 9 Zero-Hour Socially Engineered Malware Block Rate

    Googles Download Protection technology and Microsofts App Rep technologies are the reason that Chrome and IE are able to block such high percentages of SEM. Neither Chrome nor IE relies on the certain knowledge that a file is bad; rather, they block files that do not meet reputational criteria.

    Figure 10 Content Agnostic Malware Protection Breakout

    Figure 10 displays the combination of SEM-blocking technologies used by Chrome and IE. Both URL and CAMP protection methodologies can suffer from false positives; however, the more important consideration is the protection that URL reputation adds over CAMP.

    u Google buys VirusTotal

    0%

    20%

    40%

    60%

    80%

    100%

    2009 Q1 2010 Q3 2010 2011 2012 2013

    Chrome Firefox IE Safari

    10.0%

    83.2%

    4.5%

    88.5%

    16.8%

    10.6%

    73.2%

    65.8%

    0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

    2013 Chrome

    2013 IE

    2012 Chrome

    2012 IE

    URL Reputahon Applicahon Reputahon Download Protechon

  • NSS Labs Analyst Brief Evolutions In Browser Security

    11

    When a web page contains both exploits and SEM, URL reputation will protect the user from exploits, in addition to SEM. Where CAMP is the only protection mechanism, the user can still fall victim to exploits. Consequently, IEs strong use of URL reputation compared to Chromes use renders IEs SEM protection significantly more valuable.

    Combined Protection Effectiveness

    Aggregate Values

    The purpose of combining scores to arrive at a single value is to allow for the reuse of this value in conjunction with other metrics in order to select the browser that best balances selection criteria. Phishing and SEM protection, the metrics used in this brief, are a part of overall browser security. The number, severity, and longevity of exploits against a browser constitutes a metric, and security of stored passwords is yet another metric. Privacy protection capabilities, such as those discussed in the analyst brief 2013 Browser Security Comparative Analysis: Privacy, can be combined with vulnerability metrics as well as other performance metrics in order to make educated product selection decisions based on the combined performance across all aspects of interest.

    Weight A Minute

    Not all protection metrics are equal. The majority of phishing attacks will fail because the intended victim is not a customer of the targeted brand. A customer of Barclays is not going to fall victim to a phish against Wells Fargo customers. A user without a Gmail account will not surrender credentials if they receive a phish targeting Gmail.

    SEM attacks are brand agnostic and can even customize the payload for specific operating systems. A larger percentage of SEM attacks will succeed compared to phishing attacks. As such, protection against SEM is of greater importance than phishing protection.

    For both phishing and SEM, time is of the essence. Zero-hour SEM blocking is a more important metric than overall SEM blocking. Overall SEM blocking is of more significance than zero-hour phishing protection, and the mean block rate for phishing ranks lowest in the protection hierarchy.

    The data regarding the various block rates is empirical; however, the weighting of relative security values is subjective and the importance of protection metrics may vary based on other layers of protection in different environments. With the exception of clearly unrealistic weighting values, significant fluctuations in weights assigned to protection categories and test dates will not materially alter the rankings of the browsers. A scientific approach to weighting, if possible, would not alter significantly the results of the graphs. The raw data for the various NSS tests are provided in Appendix A. The formulas used to weight the scores and create the weighted figures are provided in Appendix B.

    Evaluating the Data

    Figure 11 depicts the rankings of the browsers without ranking the importance of different protection metrics: that SEM protection is significantly more important than phishing protection, or that newer tests are more relevant than older tests, and so on.

    The relative importance of protection types is only one of the variables that require weighting. The freshness of tests is critical. Old tests help assess a track record; however, browsers that have added new technologies are improperly impacted when older tests are weighted too highly. Equal value for older tests also paints an unreliable picture of browsers that are declining in protection ability.

  • NSS Labs Analyst Brief Evolutions In Browser Security

    12

    Figure 11 is not weighted, and it shows Firefox as out performing Chrome in the combined tests. In 2009, Firefox outperformed Chrome by 26 percent across the four metrics. In 2012 and 2013, Chrome eclipsed Firefox by 26 percent and 25 percent respectively. Clearly, an accurate ranking of the browser requires more recent scores to carry more weight than older results.

    Figure 11 Combined Test Results (Not Weighted)

    Leaving protection scores unmodified and assigning older tests progressively less weight, Figure 12 shows Chrome in second place with a significant lead over Firefox and Safari.

    Figure 12 Time Weighted

    Figure 13 does not weight for time; however, the different types of protection are weighted based on relative importance. The resulting graph does not adequately reflect performance improvements in Chrome with respect to SEM protection in 2012 and 2013. Weighting values can be found in Appendix B.

    37%

    46%

    49%

    79%

    0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

    Safari

    Chrome

    Firefox

    IE

    54%

    56%

    66%

    83%

    0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

    Safari

    Firefox

    Chrome

    IE

  • NSS Labs Analyst Brief Evolutions In Browser Security

    13

    Figure 13 Protection Weighted

    Figure 14 Time and Protection Weighted Scores

    Figure 14 incorporates weighting that places a higher value on newer tests and a higher value on more important protection categories.

    Figure 15 2013 Weighted Scores

    Both Figure 14 and Figure 15 display a wider performance margin between IE and Chrome, as well as between Chrome and the other browsers than does Figure 2. Figure 15 does not consider trends or track records. If current protection value, track records, and trends are considerations, then Figure 14 provides a more comprehensive picture than does Figure 15.

    27%

    36%

    39%

    80%

    0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

    Safari

    Firefox

    Chrome

    IE

    39%

    40%

    58%

    85%

    0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

    Safari

    Firefox

    Chrome

    IE

    35%

    37%

    71%

    92%

    0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

    Firefox

    Safari

    Chrome

    IE

  • NSS Labs Analyst Brief Evolutions In Browser Security

    14

    The Great Equalizer Both phishing and SEM are social engineering attacks. By definition, these are social problems, and technology has rarely solved a social problem. Technology can help to mitigate problems, but education is paramount. For users who are adept at identifying social engineering attacks, the browser adds little additional security; however, most users are not aware of the dynamics of social engineering and will fall prey to SEM even when they are able to identify many types of phishing attacks. Proper education provides the best protection against most social engineering attacks.

  • NSS Labs Analyst Brief Evolutions In Browser Security

    15

    Appendix A: Raw Data The figures in Appendix A provide the raw data from NSS testing used to create all of the figures in this analyst brief, with the exception of Figure 3, Figure 4, Figure 10, and the APWG metrics in Figure 7. The APWG metrics in Figure 3, Figure 4, and Figure 7 are derived directly from APWG published reports. The percentages used in Figure 10 are published in the 2012 and 2013 Browser Phishing Protection CARs.

    Browser 2009 2012 2013 Chrome 26% 94% 92% Firefox 80% 90% 96% IE 83% 92% 83% Safari 2% 91% 95%

    Figure 16 Mean Block Rate for Phishing

    The mean block rate for phishing reflects overall phishing detection for the duration of each test.

    Browser 2009 2012 2013 Chrome 16.00% 53.20% 81.50% Firefox 48.00% 79.20% 93.30% IE 52.00% 55.90% 73.30% Safari 2.00% 76.90% 93.40%

    Figure 17 Zero-Hour Block Rate

    The zero-hour block rate is a critical metric. The value of a higher mean block rate can be marginalized by ineffective zero-hour performance. Due to the diminishing uptimes of phishing sites, a browser with a lower mean block rate and better zero-hour response times may provide more effective protection for most users than will a browser with a better overall block rate but poorer zero-hour performance.

    Browser 2009 Q1 2010 Q3 2010 2011 2012 2013 Chrome 16% 17% 3% 13% 70% 83% Firefox 30% 29% 19% 8% 4% 10% IE 69% 85% 99% 99% 99% 100% Safari 24% 29% 11% 8% 4% 10%

    Figure 18 Mean Block Rate for SEM

    The mean block rate for SEM reflects the SEM performance for the duration of each test. Throughout a test, various browsers can fluctuate significantly in their instantaneous block rate. Histograms in NSS CARs provide additional detail.

  • NSS Labs Analyst Brief Evolutions In Browser Security

    16

    Browser 2009 Q1 2010 Q3 2010 2011 2012 2013 Chrome 25% 15% 4% 10% 67% 49% Firefox 28% 28% 18% 7% 6% 8% IE 41% 58% 89% 99% 86% 98% Safari 13% 27% 10% 6% 5% 12%

    Figure 19 Zero-Hour Block Rate for SEM

    The zero-hour block-rate is the percentage of malware each browser was already blocking when the hosting site was first discovered. Browsers with higher zero-hour protection generally provide better protection than browsers with delayed protection times.

  • NSS Labs Analyst Brief Evolutions In Browser Security

    17

    Appendix B: Weighting Formulas The following multipliers were used in calculating weighted scores.

    Protection Multipliers: Mean Block Rate for Phishing = Score * .3 Zero Hour Phishing Block Rate = Score * .5 Mean SEM Block Rate = Score * .8 Zero Hour SEM Block Rate = Score * 1

    Year Multipliers: 2009 = Score * .1 2010 = Score * .2 2011 = Score * .4 2012 = Score * .8 2013 = Score * 1

    Figure 20 displays scores weighted for date and not protection type. Figure 21 displays scores weighted for protection type but not for the dates of the test. Figure 22 displays the combined protection type and date weighting.

    Protection 2009 2010 2011 2012 2013 Mean Block Rate - Phish 10% X X 80% 100% Zero-Hour Block Rate - Phish 10% X X 80% 100% Mean Block Rate - SEM 10% 20% 40% 80% 100% Zero-Hour Block Rate - SEM 10% 20% 40% 80% 100%

    Figure 20 Time Weighting

    Protection 2009 2010 2011 2012 2013 Mean Block Rate - Phish 30% X X 30% 30% Zero-Hour Block Rate - Phish 50% X X 50% 50% Mean Block Rate - SEM 80% 80% 80% 80% 80% Zero-Hour Block Rate - SEM 100% 100% 100% 100% 100%

    Figure 21 Protection Type Weighting

    Protection 2009 2010 2011 2012 2013 Mean Block Rate - Phish 3% X X 24% 30% Zero-Hour Block Rate - Phish 5% X X 40% 50% Mean Block Rate - SEM 8% 16% 32% 64% 80% Zero-Hour Block Rate - SEM 10% 20% 40% 80% 100%

    Figure 22 Type and Year Weighting

  • NSS Labs Analyst Brief Evolutions In Browser Security

    18

    The maximum time and protection type weighted scores attainable for four protection types are as follows:

    Mean Block Rate - Phish: ((100*.03)+(100*.24)+(100*.3))/3=.19 or 19%. Zero Hour Block Rate - Phish: ((100*.05)+(100*.4)+(100*.5)/3=.3167 or 31.67%. Mean Block Rate - SEM: ((100*.08)+(100*.16)+(100*.16)+(100*.32)+(100*.64)+(100*.8))/6=.36 or 36%. Zero Hour Block Rate - SEM: ((100*.1)+(100*.2)+(100*.2)+(100*.4)+(100*.8)+(100*.1))/6=.45 or 45%.

    The maximum combined total score is therefore:

    (.19+31.67+.36+.45)/4=.3292 or 32.92%.

    To normalize to a 100 percent scale, the total weighted scores are divided by.3292.

    For Chrome, the total weighted performance for all of the tests would be calculated as follows:

    (((((((0.26*0.03)+(0.94*0.24)+(0.92*0.3))/3)+(((0.16*0.05)+(0.53*0.4)+(0.82*0.5))/3)+(((0.16*0.08)+(0.17*0.16)+(0.03*0.16)+(0.13*0.32)+(0.7*0.64)+(0.83*0.8))/6)+(((0.25*0.1)+(0.15*0.2)+(0.04*0.2)+(0.1*0.4)+(0.67*0.8)+(0.49))/6))/4)))/.3292

    Chrome - Mean Block Rate - Phish: ((.26*.03)+(.94*.24)+(.92*.3))/3=17% Chrome - Zero Hour Block Rate - Phish: ((.16*.05)+(.53*.4)+(.82*.5)/3=21% Chrome - Mean Block Rate - SEM: ((.16*.08)+(.17*.16)+(.3*.16)+(.13*.32)+(.7*.64)+(.83*.8))/6=20% Chrome - Zero Hour Block Rate - SEM: ((.25*.1)+(.15*.2) +(.04*.2)+(.1*.4)+(.67*.8)+(.49*1))/6=.19%

    Chrome - ((17%+21%+20%+19%)/4)/.3292=58%

  • NSS Labs Analyst Brief Evolutions In Browser Security

    19

    Reading List 2013 Browser Security Comparative Analysis Report: Phishing Protection. NSS Labs https://www.nsslabs.com/reports/2013-browser-security-comparative-analysis-phishing-protection

    2012 Browser Security Comparative Analysis Report: Phishing Protection. NSS Labs https://www.nsslabs.com/reports/2012-browser-security-comparative-analysis-phishing-protection

    User Education Effectiveness Can Be Measured. NSS Labs https://www.nsslabs.com/reports/user-education-effectiveness-can-be-measured

    2013 Browser Security Comparative Analysis: Privacy. NSS Labs https://www.nsslabs.com/reports/browser-security-comparative-analysis-privacy

  • NSS Labs Analyst Brief Evolutions In Browser Security

    20

    2013 NSS Labs, Inc. All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the authors.

    Please note that access to or use of this report is conditioned on the following:

    1. The information in this report is subject to change by NSS Labs without notice.

    2. The information in this report is believed by NSS Labs to be accurate and reliable at the time of publication, but is not guaranteed. All use of and reliance on this report are at the readers sole risk. NSS Labs is not liable or responsible for any damages, losses, or expenses arising from any error or omission in this report.

    3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY NSS LABS. ALL IMPLIED WARRANTIES, INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT ARE DISCLAIMED AND EXCLUDED BY NSS LABS. IN NO EVENT SHALL NSS LABS BE LIABLE FOR ANY CONSEQUENTIAL, INCIDENTAL OR INDIRECT DAMAGES, OR FOR ANY LOSS OF PROFIT, REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

    4. This report does not constitute an endorsement, recommendation, or guarantee of any of the products (hardware or software) tested or the hardware and software used in testing the products. The testing does not guarantee that there are no errors or defects in the products or that the products will meet the readers expectations, requirements, needs, or specifications, or that they will operate without interruption.

    5. This report does not imply any endorsement, sponsorship, affiliation, or verification by or with any organizations mentioned in this report.

    6. All trademarks, service marks, and trade names used in this report are the trademarks, service marks, and trade names of their respective owners.

    Contact Information NSS Labs, Inc. 206 Wild Basin Rd Building A, Suite 200 Austin, TX 78746 USA +1 (512) 961-5300 [email protected] www.nsslabs.com

    This analyst brief was produced as part of NSS Labs independent testing information services. Leading products were tested at no cost to the vendor, and NSS Labs received no vendor funding to produce this analyst brief.