annoyed users: ads and ad-block usage in the wildenric/publications/imc230-pujola... · annoyed...
TRANSCRIPT
![Page 1: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/1.jpg)
Annoyed Users:
Ads and Ad-Block Usage in the Wild
IMC’15 Tokyo, Japan
Oliver Hohlfeld
RWTH Aachen
Anja Feldmann
TU Berlin
Enric Pujol
TU Berlin
![Page 2: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/2.jpg)
2
![Page 3: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/3.jpg)
3
http://www.journalism.org/2015/04/29/digital-news-revenue-fact-sheet
Page Fair and Adobe report on Ad Blocking
![Page 4: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/4.jpg)
4
Mozilla Add-on statistics
![Page 5: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/5.jpg)
Publishers reaction
5
The Guardian
![Page 6: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/6.jpg)
6(Why can I not see BILD.de?)
![Page 7: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/7.jpg)
Acceptable ads program(non-intrusive ads)
7
White-listing within AdBlockPlus (ABP)
![Page 8: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/8.jpg)
Our work
8
Online advertisingResidential Broadband Network
Methodology based on AdBlockPlus
Characterize ad trafficAd-blocker usage
![Page 9: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/9.jpg)
9
![Page 10: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/10.jpg)
10
![Page 11: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/11.jpg)
11
URL
||ads.example.com^ $ script, image, domain=bar.com
www.bar.com
#document HTML HEAD BODY … IMG
DOM Tree
Content-Type Base URL
Is this information available in HTTP header traces?
http://ads.example.com/ad.gif
www.foo.com
Content Policy
![Page 12: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/12.jpg)
12
||ads.example.com ^ $ script, image, domain=foo.com
![Page 13: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/13.jpg)
13
||ads.example.com ^ $ script, image, domain=foo.com
HTTP logsBro IDS
![Page 14: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/14.jpg)
14
||ads.example.com ^ $ script, image, domain=foo.com
HTTP logsBro IDS
Map Web PagesReferrerMap
![Page 15: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/15.jpg)
15
||ads.example.com ^ $ script, image, domain=foo.com
Classify requestslibadblockplus + filter lists
HTTP logsBro IDS
Map Web PagesReferrerMap
![Page 16: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/16.jpg)
Limitations
16
HTTPS trafficContent-Type
HTTP logs
Referrer Map Web page
Classification Hidden ads (CSS)List sanity
![Page 17: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/17.jpg)
Ad-blocker usage indicators
17
Low ratio of ad requests Easylist updates
AdBlock Plus
Caveat: ad-free content
Any ad-blocker
Caveat: HTTPS + NATs
How to find a suitable threshold?
![Page 18: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/18.jpg)
Active measurements
Values for the complete set of Alexa top 1K
Browser Mode #HTTP ad-requests ratio
Vanilla 57.8K 16 %
Adblock Plus + all lists 48.5K <0.1 %
1828.10.2015
![Page 19: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/19.jpg)
When can we apply ad-ratio indicator?
At least 1K requests. Set threshold to 5%1928.10.2015
Vanilla: no pluginAdBP-Pa: adblockPlus + all lists
![Page 20: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/20.jpg)
Data sets
Trace RBN-1 RBN-2
Duration 4 days 15 and ½ hours
Subscribers 7.5 K 19.7 K
HTTPbytes 18.8 TB 11.4 TB
HTTPrequests 131.95 M 85.09 M
Characterize ad traffic
Assess ad-blocker prevalence
2028.10.2015
![Page 21: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/21.jpg)
How much ad-related traffic?
2128.10.2015
Ad-related traffic
EasyList and derivatives (55 %) EasyPrivacy (35 %) Acceptable ads (10 %)
0 % 18 % 100 %
Black-listed (57%)
“Loose filters” @@||gstatic.com^$document
![Page 22: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/22.jpg)
How many ABP-capable browsers?
Requests: 50 % (all)72 % (ads)
2228.10.2015
500K pairs<IP, UserAgent>
1.6K unique UserAgent strings 9.2 K users
Heavy hitters
MobileFirefox Chrome Safari IE
33 % 24 % 15 % 21 % 7 %
Known browsers
![Page 23: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/23.jpg)
23
Ad Ratio EasyList
✘ ✘
✘ ✔
✔ ✘
✔ ✔
28.10.2015
Browsers with no evidence of an adblocker: 46% of the ad traffic
![Page 24: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/24.jpg)
Inconsistent indicatorsa) various users in the same household
2428.10.2015
Ad Ratio EasyList
✘ ✘
✘ ✔
✔ ✘
✔ ✔
![Page 25: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/25.jpg)
Inconsistent indicatorsa) different plugin, or b) ad-free content
2528.10.2015
Ad Ratio EasyList
✘ ✘
✘ ✔
✔ ✘
✔ ✔
![Page 26: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/26.jpg)
Evidence of Adblock Plus for 22% of the users Still generate ad traffic!
2628.10.2015
Ad Ratio EasyList
✘ ✘
✘ ✔
✔ ✘
✔ ✔
?
![Page 27: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/27.jpg)
27
Most AdBlockPlus users do not change default settings
82% 11 % 7%
EasyPrivacyAcceptable ads
EasyList
No ad blocker
EasyPrivacy
Acceptable ads
99%
87%
ABP users
90%
70%
How do users configure ABP?AGGREGATED
![Page 28: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/28.jpg)
Summary
Can we use existing functionality to classify ad traffic?
Yes, we can! (but limitations apply)
2828.10.2015
![Page 29: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/29.jpg)
Summary
Can we use existing functionality to classify ad traffic?
Yes, we can! (but limitations apply)
Which traffic is potentially affected by Adblock Plus?
~18% requests and ~1% HTTP bytes
Google, CDNs, Cloud Providers and dedicated ASes
10% of the ad-related traffic is white listed
2928.10.2015
![Page 30: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/30.jpg)
Summary
Can we use existing functionality to classify ad traffic?
Yes, we can! (but limitations apply)
Which traffic is potentially affected by Adblock Plus?
~18% requests and ~1% HTTP bytes
Google, CDNs, Cloud Providers and dedicated ASes
10% of the ad-related traffic conforms Acceptable Ads
Can we measure how end users adopt ad-block plugins?
Yes, in our traces ~20% of users
Out-of-the-box i.e., acceptable ads / no EasyPrivacy3028.10.2015
![Page 31: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/31.jpg)
Thank you!
Can we use existing functionality to classify ad traffic?
Yes, we can! (but limitations apply)
Which traffic is potentially affected by Adblock Plus?
~18% requests and ~1% HTTP bytes
Google, CDNs, Cloud Providers and dedicated ASes
10% of the ad-related traffic conforms Acceptable Ads
Can we measure how end users adopt ad-block plugins?
Yes, in our traces ~20% of users
Out-of-the-box i.e., acceptable ads / no EasyPrivacy3128.10.2015
![Page 32: Annoyed Users: Ads and Ad-Block Usage in the Wildenric/publications/imc230-pujolA... · Annoyed Users: Ads and Ad-Block Usage in the Wild IMC’15 Tokyo, Japan Oliver Hohlfeld RWTH](https://reader036.vdocuments.mx/reader036/viewer/2022062922/5f0756c27e708231d41c7e19/html5/thumbnails/32.jpg)
3228.10.2015