Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

A Longitudinal View of Dual-stacked Websites−Failures, Latency and Happy Eyeballs

Vaibhav BajpaiTU Munich

IETF 99Prague, CZ

Joint work with

Jürgen SchönwälderJacobs University Bremen, Germany

Steffie Jacob EravuchiraSamknows Limited, London, UK

Sam CrawfordSamknows Limited, London, UK

July 2017

1 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Motivation

▶ Literature focus largely on IPv6 adoption.▶ Very little work on measuring IPv6 performance.▶ This study closes the gap. 2009 2010 2011 2012 2013 2014 2015 2016 2017

0%5%10%15%

Google IPv6 Adoption

shaded region represents the duration of the longitudinal study.

∼100 dual-stacked SamKnows probes (∼66 different origin ASes)

NETWORK TYPE #

RESIDENTIAL 78

NREN / RESEARCH 10

BUSINESS / DATACENTER 08

OPERATOR LAB 04

IXP 01

RIR #

RIPE 60

ARIN 29

APNIC 10

AFRINIC 01

LACNIC 01

2 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Motivation

-1.0%0.0%1.0%2.0%3.0%

Google IPv6 AdoptionNative IPv6

2008 2009 2010 2011 2012 2013 2014 2015 2016-0.02%0.0%0.02%0.04%0.06%

Weekends - Weekdays

Teredo/6to4

▶ The drift of IPv6 penetration is increasing.

Country Distribution

Japan: 29.63 %Japan: 29.63 %

Peru: 20.37 %Peru: 20.37 %

Malaysia: 3.70 %Malaysia: 3.70 %

United States: 1.85 %United States: 1.85 %

Brazil: 16.67 %Brazil: 16.67 %

Pakistan: 1.85 %Pakistan: 1.85 %

Argentina: 3.70 %Argentina: 3.70 %

Kenya: 1.85 %Kenya: 1.85 %

Australia: 3.70 %Australia: 3.70 %

South Africa: 1.85 %South Africa: 1.85 %

Sri Lanka: 3.70 %Sri Lanka: 3.70 %

Bulgaria: 1.85 %Bulgaria: 1.85 %

Zambia: 1.85 %Zambia: 1.85 %

Taiwan: 1.85 %Taiwan: 1.85 %

Botswana: 1.85 %Botswana: 1.85 %

Canada: 3.70 %Canada: 3.70 %

Highcharts.com

▶ Prefixes blacklisted by Google over IPv6 (2017)

2010 2011 2012 2013 2014 2015 2016 20170.0%5.0%10.0%15.0%20.0%

W6D

W6LD

ALEXA 1M with AAAA entries

0% 10% 20% 30% 40%Websites

top 0.1Ktop 1Ktop 10Ktop 100Ktop 1000K

40.025.7

20.417.4

12.1

▶ Cloudflare Effect (shaded region)▶ The dent created by Cloudflare > W6D (or W6LD).▶ Cloudflare added AAAA entries for all websites [1].

A CDN plays a leading role in technology adoption andshifting significant traffic overnight over IPv6.

3 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Research Contribution

▶ Complete Failures

▶ Partial Failures

▶ Failures silently exist; clients do not notice them due to IPv4 fallback.

▶ Can websites with partial failures be deemed IPv6-ready?

▶ Quantification of failures helpful for upcoming IPv6-only networks.

▶ Latency

▶ Happy Eyeballs

This is the first study to provide a longitudinal view (4 years)of failures and performance of dual-stacked websites.

4 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Complete Failures

2010 2011 2012 2013 2014 2015 2016 20170.0%10.0%20.0%30.0%40.0%50.0%

W6D

W6LD

ALEXA 1M with AAAA entries

HTTP Failure

▶ Failures reduced from 40% (2009) to 3% today.

0.1K 1K 10K 100K 1000KALEXA Rank

0.00.20.40.60.81.0

CDF

Failing AAAA Websites

4.3K[M

ar '

17]

▶ 88% failing websites rank > 100K.▶ 1% rank < 10K, six websites rank < 300.

100101102103

www.bing.com

102

103www.detik.com

100101102103

www.engadget.com

102

103www.nifty.com

100101102103104

www.qq.com

Jan2013

Jan2014

Jan2015

Jan2016

Jul Jul Jul102

103www.sakura.ne.jp

IPv6 IPv4

TCP Connect Times (ms)

Metrics should account for changes in IPv6-readiness.

5 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Complete Failures | WL6D websites

▶ ∼3K websites participated on the W6LD (2012).

▶ Promise to permanently enable production-ready IPv6 on the Internet.

W6LD websites −▶ 8% (259) do not have A or AAAA entries in DNS.

W6LD websites (with A and AAAA entries)−

▶ 1% (36) TCP timeout over both AF.

▶ 8% (253) TCP timeout over IPv6.

1K 10K 100K 1M 10M 100MALEXA Rank

0.00.20.40.60.81.0

CDF

Failing W6LD Websites

253

▶ 3% failing websites rank < 10K.

▶ 75% rank > 100K, 61% rank > 1M.

6 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Partial Failures

ALEXA top 100 websites with AAAA entries.

▶ 27% show some rate of failure over IPv6.▶ 9% exhibit more than 50% failures over IPv6.

0 20 40 60 80 100Success Rate (%)

0.00.20.40.60.81.0

CDF

IPv6 (100)IPv4 (100)

▶ Limiting to root webpage can lead tooverestimation of IPv6 adoption numbers

▶ Unclear whether websites with partialfailures can be deemed IPv6-ready

▶ ISOC now supporting [2] development oftools that identify such partial failures

# Webpage Success Rate (%) W6LDIPv6(↓) IPv4

01 www.bing.com 0 100 302 www.detik.com 0 100 303 www.engadget.com 0 100 304 www.nifty.com 0 100

05 www.qq.com 0 100

06 www.sakura.ne.jp 0 100

07 www.flipkart.com 09 99 308 www.folha.uol.com.br 13 100

09 www.aol.com 48 100 3

10 www.comcast.net 52 100 311 www.yahoo.com 72 100 312 www.mozilla.org 84 100 313 www.orange.fr 86 100 314 www.seznam.cz 89 100 315 www.mobile.de 90 100 316 www.wikimedia.org 90 100

17 www.t-online.de 93 100 318 www.free.fr 95 100

19 www.usps.com 95 100

20 www.vk.com 95 100 321 www.wikipedia.org 95 100 322 www.wiktionary.org 95 100

23 www.elmundo.es 96 100 324 www.uol.com.br 96 100 325 www.marca.com 97 100 326 www.terra.com.br 98 100 327 www.youm7.com 99 100

7 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Partial Failures | Root Cause Analysis

0 30 60 90www.youm7.com (1%)

www.terra.com.br (2%)www.marca.com (3%)www.uol.com.br (4%)www.elmundo.es (4%)

www.wiktionary.org (5%)www.wikipedia.org (5%)

www.vk.com (5%)www.usps.com (5%)www.free.fr (5%)

www.t-online.de (7%)www.wikimedia.org (10%)

www.mobile.de (10%)www.seznam.cz (11%)www.orange.fr (14%)

www.mozilla.org (16%)www.yahoo.com (28%)

www.comcast.net (48%)www.aol.com (52%)

www.folha.uol.com.br (87%)www.flipkart.com (91%)www.sakura.ne.jp (100%)

www.qq.com (100%)www.nifty.com (100%)

www.engadget.com (100%)www.detik.com (100%)www.bing.com (100%)

Network LevelCURLE_OKCURLE_COULDNT_RESOLVE_HOSTCURLE_COULDNT_CONNECTCURLE_OPERATION_TIMEDOUTCURLE_GOT_NOTHINGCURLE_RECV_ERROR

0 30 60 90Contribution (%)

Content Level

*/css*/html*/javascript, */json*/octet-stream*/plain*/rdf*/xmlimage/*

0 30 60 90

Service Level

SAME ORIGINCROSS ORIGIN

Website failing over IPv6

▶ Failures due toDNS resolutionerror onimage/*,

*/javascript,

*/json and

*/css content.

▶ Failures silently exist; clients do not notice them due to IPv4 fallback.▶ Identification of operational issues relevant for upcoming IPv6-only networks

8 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Partial Failures | Root Cause Analysis

0 30 60 90Contribution (%)

www.youm7.com (1%)www.terra.com.br (2%)

www.marca.com (3%)www.uol.com.br (4%)www.elmundo.es (4%)

www.wiktionary.org (5%)www.wikipedia.org (5%)

www.vk.com (5%)www.usps.com (5%)www.free.fr (5%)

www.t-online.de (7%)www.wikimedia.org (10%)

www.mobile.de (10%)www.seznam.cz (11%)www.orange.fr (14%)

www.mozilla.org (16%)www.yahoo.com (28%)

www.comcast.net (48%)www.aol.com (52%)

www.folha.uol.com.br (87%)www.flipkart.com (91%)www.sakura.ne.jp (100%)

www.qq.com (100%)www.nifty.com (100%)

www.engadget.com (100%)www.detik.com (100%)www.bing.com (100%)

*.youm7.com*.terra.com.br*.marca.com*.uol.com.br*.elmundo.es*.wiktionary.org*.wikipedia.org*.vk.com*.usps.com*.free.fr*.t-online.de*.wikimedia.org*.mobile.de*.seznam.cz*.orange.fr*.mozilla.org*.yahoo.com*.comcast.net*.aol.com*.uol.com.br*.flipkart.com*.sakura.ne.jp*.qq.com*.nifty.com*.engadget.com*.detik.com*.bing.com

SAME ORIGIN

▶ 12% of websites have more than 50% webpage elementsthat belong to the same origin source and fail over IPv6.

▶ CDN infrastructure does not have IPv6 turned on bydefault for all same-origin webpage elements.

# Webpage Same Origin (↓)

01 www.bing.com 100%

02 www.detik.com 100%

03 www.engadget.com 100%

04 www.nifty.com 100%

05 www.usps.com 100%

06 www.qq.com 100%

07 www.sakura.ne.jp 100%

08 www.comcast.net 85%

09 www.yahoo.com 83%

10 www.terra.com.br 74%

11 www.marca.com 70%

12 www.wikimedia.org 65%

13 www.elmundo.es 37%

14 www.vk.com 31%

15 www.t-online.de 30%

16 www.youm7.com 24%

17 www.wiktionary.org 22%

18 www.wikipedia.org 22%

19 www.free.fr 13%

20 www.folha.uol.com.br 12%

21 www.mozilla.org 7%

22 www.uol.com.br 7%

23 www.mobile.de 7%

24 www.aol.com 5%

25 www.orange.fr 5%

26 www.seznam.cz 4%

27 www.flipkart.com 1%

9 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Partial Failures | Root Cause Analysis

0 30 60 90Contribution (%)

www.youm7.com (1%)www.terra.com.br (2%)

www.marca.com (3%)www.uol.com.br (4%)www.elmundo.es (4%)

www.wiktionary.org (5%)www.wikipedia.org (5%)

www.vk.com (5%)www.usps.com (5%)www.free.fr (5%)

www.t-online.de (7%)www.wikimedia.org (10%)

www.mobile.de (10%)www.seznam.cz (11%)www.orange.fr (14%)

www.mozilla.org (16%)www.yahoo.com (28%)

www.comcast.net (48%)www.aol.com (52%)

www.folha.uol.com.br (87%)www.flipkart.com (91%)www.sakura.ne.jp (100%)

www.qq.com (100%)www.nifty.com (100%)

www.engadget.com (100%)www.detik.com (100%)www.bing.com (100%)

CROSS ORIGIN

*.adition.com*.ajax.googleapis.com*.aolcdn.com*.cimcontent.net*.creativecommons.org*.d5nxst8fruw4z.cloudfront.net*.demdex.net*.dmtry.com*.doubleclick.net*.el-mundo.net*.elmundo.es*.expansion.com*.f.i.uol.com.br*.flixcart.com*.globaliza.com*.images1.folha.com.br*.imedia.cz*.imguol.com*.imguol.com.br*.interactivemedia.net*.ioam.de*.jsuol.com.br*.leguide.com

*.ligatus.com*.mail.ru*.mozilla.net*.navdmp.com*.netbiscuits.net*.omtrdc.net*.optimizely.com*.outbrain.com*.proxad.net*.quantserve.com*.sblog.cz*.scorecardresearch.com*.szn.cz*.tag.navdmp.com*.telva.com*.theadex.com*.toi.de*.trrsf.com*.unidadeditorial.es*.voila.fr*.woopic.com*.www1.folha.com.br*.xiti.com

▶ Third-party advertisements(*.doubleclick.net)

▶ Analytics(*.scorecardresearch.com,*.quantserve.com)

▶ User-centric content(*.facebook.com,*.ajax.googleapis.com)

▶ Static content(*.wikimedia.org,*.creativecommons.org)

▶ Enabling IPv6 on few cross cross-origin sources (creativecommons.org,doubleclick.net) will help reduce partial failure of multiple websites.

10 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Latency | Websites

∆sa(u) = t4(u) − t6(u)

where t(u) is the time taken to establish TCP connection to website u.

▶ ISPs in early stages of IPv6 deployment shouldensure their CDN caches are dual-stacked.

−150−100−50050

TCP Connect Times [∆sa (ms)]

www.bing.comwww.facebook.com

www.wikipedia.orgwww.youtube.com

2013 2014 2015 2016 2017−60−40−20020

www.blogspot.*www.google.*

www.netflix.comwww.yahoo.com

▶ TCP connect times to popular websites over IPv6 have considerably improved over time.▶ Inflated latency over IPv6 was due to missing content caches over IPv6

11 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Latency | Websites - Who connects faster?

ALEXA top 10K websites (as of Jan 2017):

▶ 40% are faster over IPv6.

▶ 94% of the rest are at most 1 ms slower.

▶ 3% are at least 10 ms slower.

▶ 1% are at least 100 ms slower. −1.0 −0.5 0.0 0.5 1.0∆sa (ms)

0.0

0.2

0.4

0.6

0.8

1.0

CDF

netflixyahoo

google

linkedinmicrosoft facebook

wikipedia

cloudflare heise

openstreetmap

ALEXA (10K)

[01/

2017

]

∆sa(u) = t4(u) − t6(u)

▶ Relevant for content providers to get insights on how their service delivery compares over IPv6.

12 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Happy Eyeballs

▶ Only ∼1% of samples aboveHE timer value > 300 ms

Websites where HE prefers IPv6 −

▶ A 300 ms HE timer value leaves2% chance for IPv4.

▶ 99% of top 10K ALEXA preferIPv6 98% of time.

10-2 10-1 100 101 102 103 104TCP Connect Times (ms)

0.00.20.40.60.81.0

CDF

300

ms

IPv6 (462K)IPv4 (462K)

['13

- '

17]

96% 97% 98% 99% 100%0.00.20.40.60.81.0

CCDF

PROBES (80)ALEXA (10K)

['13

- '

17]

Preference (300 ms)

13 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Happy Eyeballs

Samples where HE prefers IPv6 −

▶ HE prefers slower IPv6connections 90% of the time.

▶ HE timer of 150 ms maintainssame IPv6 preference levels.

▶ We get margin benefit of 10%because timer cuts early.

−40 −30 −20 −10 0 10∆sa (ms)

0.00.20.40.60.81.0

CDF

1% 2% 7%30%

93%99%

462K

['13

- '

17]

0 50 100 150 200 250 300HE timer (ms)

0%20%40%60%80%100%

Preference

150 ms

ALEXA (10K)

['13 - '17]

▶ RFC 6555 should have used 150 ms timer. Measurements should inform protocol engineering.▶ Drive an RFC 6555 update with operational experience within the IETF.

14 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

This work would not have been possible without these amazing people −

■ Jürgen Schönwälder ■ Jörg Ott ■ Sam Crawford ■ Jamie Mason ■ Steffie Jacob Eravuchira ■ Philip Eardley ■Trevor Burbridge ■ Andrea Soppera ■ Olivier Bonaventure ■ Dan Wing ■ Andrew Yourtchenko ■ StephenStrowes ■ Eric Vyncke ■ Arthur Berger ■ Karen Davis ■ Fred Baker ■ Aiko Pras ■ Al Morton ■ Alex Buie ■Alexandru Dumitru ■ Andrew Barnes ■ Antonio Prado ■ Antonio Querubin ■ Arash Naderpour ■ Bart Van DerVeer ■ Bayani Benjamin Lara ■ Bill Walker ■ Brandon Ross ■ Chiang Fong Lee ■ Chris Baker ■ ChristianKaufmann ■ Clinton Work ■ Daniel Karrenberg ■ Dario Ercole ■ Dustin Grant ■ Edi SUC ■ Edoardo Martelli ■Egil-Andre Oelschlagel ■ Emile Aben ■ Erik Taraldsen ■ Espen Pederson ■ Faruk Sejdic ■ Filip De Turck ■Frank Bulk ■ Fuminori Tany Tanizaki ■ Heinrich Stamerjohanns■ Herman Hermansen ■ Hugo Slabbert ■ JamesCutler ■ Jan Zorz ■ Jason Fesler ■ Jasper Rappard ■ Javier Henderson ■ Jens Hoffmann ■ Jesse Sowell ■ JoelMaslak■ John Ioannidis■ Jon Thompson■ Jordi Palet■ Josh Hoge■ Juan Cerezo ■ Juan Cordero■ KawashimaMasanobu ■ Kinga Lipskoch ■ Kris Lambrechts ■ Kristijan Lecnik ■ Krunal Shah ■ Lucas do Amaral Saboya ■Luka Manojlovic ■ Marco Hogewoning ■ Marco Sommani ■ Marian Neagul ■ Martin Neitzel ■ Masaki Tagawa■ Mat Ford ■ Mathew Newton ■ Matthieu Bouthors ■ Michael Carey ■ Michael Richardson ■ Michael VanNorman ■ Mikael Abrahamsson ■ Mike Taylor ■ Mircea Suciu ■ Nick Chettle ■ Nishal Goburdhan ■ Ole Troan■ Owen DeLong ■ Pedro Tumusok ■ Per Olsson ■ Peter Bulckens ■ Philip Homburg ■ Philip Smith ■ PhillipRemaker ■ Radek Krejci ■ Richard Patterson ■ Robert Kisteleki ■ Ryan Rawdon ■ Saba Ahsan ■ Sergey Sarayev■ Steinar Haug■ Steve Bauer■ Tero Marttila■Thibault Cholez■Thomas Schafer■ Tim Chown■ Tim Coote■Tim Martin ■ Tobias Oetiker ■ Torbjorn Eklov ■ Trond Hastad ■ Uros Gaber ■ Vesna Manojlovic ■ VladUngureanu ■ Wouter de Vries ■ Yasuyuki Kaneko

15 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

Impact

▶ A Longitudinal View of Dual-Stacked Websites −▶ Failures [3] [CNSM ′16]

▶ Latency [4] and [NETWORKING ′15]

▶ Happy Eyeballs [5] [ANRW ′16]

▶ Relevance:▶ Network operators in early stages of IPv6 deployment.▶ Content providers to see how their service delivery over IPv6 compares to IPv4.▶ Drive related standards work in the IETF.

www.vaibhavbajpai.com

bajpaiv@in.tum.de | @bajpaivaibhav

16 / 16

Motivation

Contribution

Complete Failures

Partial Failures

Latency

Happy Eyeballs

Acknowledgements

Q/A

References

[1] “98.01% of sites on Cloudflare now use IPv6,”https://blog.cloudflare.com/98-percent-ipv6, [Online; accessed15-Apr-2017].

[2] “NAT64 Check,” nat64check.ipv6-lab.net, [Accessed 15-Apr-2017].

[3] S. J. Eravuchira, V. Bajpai, J. Schönwälder, and S. Crawford,“Measuring Web Similarity from Dual-stacked Hosts,” ser.Conference on Network and Service Management, 2016, pp. 181–187.[Online]. Available: http://dx.doi.org/10.1109/CNSM.2016.7818415

[4] V. Bajpai and J. Schönwälder, “IPv4 versus IPv6 - who connectsfaster?” ser. IFIP Networking Conference, 2015, pp. 1–9. [Online].Available: http://dx.doi.org/10.1109/IFIPNetworking.2015.7145323

[5] ——, “Measuring the Effects of Happy Eyeballs,” ser. AppliedNetworking Research Workshop, 2016. [Online]. Available:http://dl.acm.org/citation.cfm?id=2959429

16 / 16