using traffic shaping to combat spam
DESCRIPTION
Using Traffic Shaping to Combat Spam. David Cawley, Senior Engineer December 12th, 2007. Overview. Evolution of E-mail & Spam Spamonomics SMTP Multiplexing Traffic Shaping Asynchronous IO Passive OS Fingerprinting. The Dawn of E-mail. 1965 MIT shared mainframe 1971 The @ symbol - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/1.jpg)
Using Traffic Shaping to Combat Spam
David Cawley, Senior Engineer
December 12th, 2007
![Page 2: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/2.jpg)
Overview
1. Evolution of E-mail & Spam
2. Spamonomics
3. SMTP Multiplexing
4. Traffic Shaping
5. Asynchronous IO
6. Passive OS Fingerprinting
![Page 3: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/3.jpg)
The Dawn of E-mail
• 1965 MIT shared mainframe
• 1971 The @ symbol
• 1976 Queen of England sends an e-mail
• 1982 IETF RFC821/822
• 1989 Lotus Notes released (35k copies sold)
• 1996 Microsoft Internet Mail 1.0
• 2001 IETF RFC2821/2822
![Page 4: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/4.jpg)
Attempts to secure...
• SMTP is inherently insecure
• SMTP-Auth/TLS
• SPF
• Sender-ID
• Why it didn't stop spam
![Page 5: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/5.jpg)
The Evolution of Spam
• 1978 The first spam
• 1988 Usenet cross-posting
• 1993 “spam” coined as a name
• 1997 Open Relays abused
• 2000 Birth of Nigerian spam
• 2001 Formail exploit
• 2003 Sobig virus sends spam
![Page 6: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/6.jpg)
The Evolution of Spam
• 2003 CAN-SPAM act
• 2004 Bill gates prediction & botnets
• 2005 Image spam, Ascii art
• 2006 Animated images, flash, pdf
• 2007 mp3, excel, p2p botnets
![Page 7: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/7.jpg)
The escalating spam problem
Source: spamnation.info/stats
The good old days.
![Page 8: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/8.jpg)
Spammer Economics
• 0.02% people click and buy [source: NY Times]
• Average filter effectiveness is 90%
– 1/10 of spam messages get through
• Improve effectiveness to 95%
– 1/20 of spam messages get through
• Spammer Solution?
– Double spam volume
– Same profit
![Page 9: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/9.jpg)
Traditional Filtering
• MD5's, Fuzzy Signatures, Bayesian
• Header Regex, RBL's, URL Lists, Grey Listing
• Problems
– Obfuscation Techniques
– Formats – html, image, pdf, doc, xls, ole, mp3..
– Zombies, Botnets
![Page 10: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/10.jpg)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 300
100000
200000
300000
400000
500000
600000
700000
800000
How often do we see a unique Botnet IP?
The Number of Unique IP's versus the number of times reported
# Times Reported
# U
niq
ue
Bo
tne
t IP
's
![Page 11: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/11.jpg)
SMTP Multiplexing
• Transparent SMTP Proxy
• Connection Pooling
• Insulates the MTA
• Avoids delay of legitimate mail
• High Concurrency
– Up to 10,000 simultaneous connections
![Page 12: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/12.jpg)
12
![Page 13: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/13.jpg)
Traffic Shaping
• What can we do?
• Provide a Quality of Service
• Reputation Network
• Throttle unknown senders
• Fast track legitimate senders
![Page 14: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/14.jpg)
![Page 15: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/15.jpg)
![Page 16: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/16.jpg)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0 50 100 150 200 250 300 350 400 450
Perc
enta
ge o
f Con
necti
ons S
till C
onne
cted
Time (Seconds)
Spammers are Less Patient than Legitimate Senders
Spammers
Legitimate Senders
16
![Page 17: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/17.jpg)
![Page 18: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/18.jpg)
Does Sendmail Throttle?
ratecontrol
ConnectionRateThrottle
conncontrol
![Page 19: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/19.jpg)
Asynchronous IO
• Non-Blocking front end
• Blocking Back-end
• Event driven
• Finite State Machine
• Management of Resources
![Page 20: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/20.jpg)
![Page 21: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/21.jpg)
Passive OS Fingerprinting
1.Look at IP packet data
2.Determine the Operating System
3.Decision to Throttle
![Page 22: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/22.jpg)
OS Comparison
Delivered
Windows
Linux
FreeBSD
Solaris
Novell
HP
NetCache
Not delivered
Windows
Linux
FreeBSD
Solaris
Novell
HP
NetCache
![Page 23: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/23.jpg)
Conclusions
1.Spamming is driven by economics
2.Botnet operators need to make money
3.Slowing down spam makes it go away
![Page 24: Using Traffic Shaping to Combat Spam](https://reader035.vdocuments.mx/reader035/viewer/2022062408/56813877550346895da02613/html5/thumbnails/24.jpg)
Nick Shelness, Former CTO, Lotus:“I am able to report that I have been running an instance of
TrafficControl in my own network for four months, and that it has reduced the volume of spam hitting my boundary MTAs on most days
by approximately 95%.”
+1-778-785-6143
www.mailchannels.com