types of spam filters

8/8/2019 Types of Spam Filters

1/5

TYPES OF SPAM FILTERS

1.Content Based Spam Filtering

These have always been the traditional types of spam filters. These simply analyse the

message subject, headers and content looking for "Kill" words or phrases, or other indicatorsof spam.Whenever an undesirable message gets through to your inbox, you simply create a new filter

by choosing certain words, or phrases from the message that indicate it is spam.Over the years, spammers have been aware that their messages were being killed by thesecontent filters and have resorted to ever more desperate tricks to try to fool the content filters.This would explain why you get so much mail for "Vi@gra", "Mort.gage", "L|0|a|n|$" etc...

This practice has become so prevalent nowadays that older content-based filters have beenperforming less well. However, some of the more modern offerings have the ability toperform "wildcard" searches and at least one has the ability to "see through" the spammer'sattempts at "obfuscating" the words/phrases such as in the examples above and even

recognise that these attempts are being made - therefore indicating that it is spam.

At the end of the day, the spammer has to get his or her message across, and the only waythey have to do that in an email is with the written word. A lot of spam messages now arevirtually illegible because of their attempts to fool content-based filters.

Pros:

y Flexible. You can easily tailor the filtering to the exact type of spam message youhave to deal with and, just as importantly, not to baulk at words or phrases that youuse daily in your business or with your friends.

Cons:

y Require more "hands on" tuning and maintenance. As spammers resort to new tricksto foil the filters, or new products get advertised, extra filters have to be created todeal with them.

2.Bayesian Based Filters

Born in London 1702, the son of a minister, Thomas Bayes developed a formula whichallowed him to determine the probability of an event occurring based on the probabilities oftwo or more independent evidentiary events.

Bayesian filters are filters that are based on this theory.Bayesian filters have to be "trained" from known "good" and "bad" e-mails. During trainingthey extract "tokens" (separate words) and store them in a database.


2/5

When analysing a new message, the message is split into tokens and each token is given a

value according to the following criteria:

y The frequency of the token in spam messages that the filter has been trained ony The frequency of the token in good messages that the filter has been trained ony The number of spam messages the filter has been trained ony The number of good messages the filter has been trained on

From applying Bayes' formula to these results, a value is extracted that gives the probabilityof this message being spam or not. This value is often called "spamicity".

Some current Bayesian based filters are returning very impressive detection rates with

minimum false positives or false negatives.

Pros:

y Require less maintenance than other filters. Once the engine has been "trained", theypretty much look after themselves.

y They automatically adapt to shifting trends in spam. Because Bayesian filters continueto learn from newly arrived messages they will naturally adapt to shifting trends.

y Will automatically adapt to the particular user's usual e-mails. If a user is, forinstance, a loan officer, then messages that repeatedly mention loans won't necessarily

be identified as spam.y Good record of minimum false positives.

Cons:

y Filtering is only as good as the messages on which they are "trained". Many filtersbased on this technology come "pre-trained", but obviously not on your type ofmessages. All will require some time before they reach optimum filtering ability.

y Has the potential to be fooled by diluting the spam message with enough obviouslyinnocent words.

3. Whitelist/Blacklist Filters

These are very basic types of filters which nowadays are rarely used on their own, but are stillused as part of an integrated filtering system comprising some of the other methods shownhere.

Whitelist filters will not accept e-mail from any address unless it is on a list of known "good"e-mail addresses.

Blacklist filters, conversely, will allow messages from any address unless the address is on alist of known "bad" sources.

Blacklists can be stored and administered on a local system or referenced via the internet.Blacklists available on the internet are referred to as "RBLs", orRealtimeBlackhole Lists.


3/5

Pros:

y Whitelists are guaranteed to stop e-mail from unwanted sources.y Properly maintained blacklists should result in zero false positives.

Cons:

y Whitelists are a drastic measure with very little flexibility.y Sometimes the people that compile RBLs - the realtime blacklists available on the

internet put entire ranges of IP addresses on their blacklist even though previousabuse occurred only an a certain part of that range. This results in "collateral damage"

- the situation where innocent people get blocked as a by-product of stopping thespammer and is the subject of much contention.

4.Challenge/Response Filters

Challenge/Response filters are characterised by their ability to automatically send a response

to an unknown sender asking them to take some further action to ensure their message will bereceived. This is often referred to as a "Turing Test" - named after a test devised by Britishmathematician Alan Turing to determine if machines could think.

Recent years have seen the appearance of some internet services which automatically performthis Challenge/Response function for the user and require the sender of an e-mail to visit theirweb site to facilitate the receipt of their message.

Critics of this system claim it to be too drastic a measure and sends a message that "my timeis more important than yours" to the people trying to communicate with you.

While this may be true, it is our opinion that it is a valid measure providing that the challengeis not sent as a matter of course, but only once a message has been analysed and deemed to be

questionable.

For some low traffic e-mail users though, this system alone may be a perfectly acceptablemethod of completely eliminating spam from their inbox - one step above the "Whitelist"system outlined above.

5.Community Filters

These types of filters work on the principal of "communal knowledge" of spam. These typesof filters communicate with a central server. When a user receives a message that is spam,they simply mark it as such. This information is posted to the central server where a

"fingerprint" of the message is added to the database. When enough people have "voted" themessage as spam, it will be blocked from user's inboxes in the future.

Pros:

y Easy to set up and very minimal administration.y "Feel Good Factor" - knowing that by marking a message as spam, you are preventing

it from being delivered to thousands of others.


4/5

Cons:

y Before enough votes are cast, somebody will be getting the spam messages.y One person's idea of spam may not be another's - consequently, some innocent mails

may be blocked by over zealous people preventing their delivery (false positives).y Some spammers will slightly change each message sent so that the "fingerprint" of the

message is different, meaning it may not be recognised as the previous messageblocked.

6. Filter Placement

One final distinguishing factor to consider between spam filters is their actual placement.There are three main schemes for filter placement:

y Filters that Integrate with your e-mail client - Many modern spam filters willintegrate with popular e-mail clients, such as Outlook or OutlookExpress.

Pros:

o Minimal impact on your normal e-mail reading habits. Spam messages areusually simply moved to a "Junk Mail" folder where they can be reviewedand/or deleted.

Cons:

o Ties you to your current e-mail client.o Inflexible - Often gives you limited choices as to your alerting level. For

instance, when running Microsoft Outlook with an integrated spam filter,whenever a spam message arrives, you still get an alert that a new message hasarrived. You have to go into the Outlook interface to confirm that the newlyarrived message was spam and not an important e-mail.

We have been unsuccessful in trying to get Outlook to create either a differentaudible alert between good and bad messages, or only to alert on the arrival of"good" messages since all messages go into the Inbox before they are actedupon by the filter and moved to a separate folder.This will result in either ignoring a new message that arr ives, or continually

being disturbed only to find that the new message is spam.

y Filters that act as a "proxy" between the mail server andyour e-mail client -These filters run in the background on your desktop and periodically poll your e-mailserver, retrieve the messages found and act on them before they reach your normal e-mail client.

y Pros:o Flexibility - usually have more control over the messages on your server and

can mark, move or even delete messages before they are seen by your normale-mail client.

o They do not tie you to any particular e-mail client.o Security - they represent another layer between the internet and your e-mail

client. They usually will not run any applications or run scripts found in the e-mail message.


5/5

Cons:

o Impose a change on your normal e-mail viewing habits. Effective use of theseinvolve turning off auto-checking on your normal e-mail client so that the

proxy has a chance to work on the server first.o E-mail account information will need to be set up in the filter as well as in

your normal e-mail client.

y Server Based Filters - These are usually only used in a corporate, or businessenvironment rather than in the home. All e-mail arrives at a central server where it isfiltered by the server-based filter and individual users collect their messages on theirdesktop from the central server.

Pros:o Central management of all e-mail filtering rules ensuring consistency across

the network.o Individual users have no little or no responsibilities for spam management,

freeing them to be more productive in their work.

Cons:

o Usually require more maintenance and require the presence and time of anexperienced network administrator to manage the filter.

o Often more expensive.

types of spam filters

Documents