Here you will find general interest articles on topics that matter to me.

Spammers Will Always Win

The Sad Facts

I have been using the internet for a long time now, and I have had an email address for most of that time. It was almost a mark of recognition, of being out there, when I started to get spam - but now I am fed up with spam.

I estimate that I spend approximately 20% of my personal internet related time either dealing with spam or researching better ways to avoid or identify it. I am using a mixture of solutions at the moment, a bayesian filter, black lists, honeypot addresses and mail rules; but none of them work 100%. Combining all these techniques together I can achieve 99.9% success. So why will the spammers always win?

Home · Contact · Blog · General Interest · Software · JHosts · Gos4j · © Hugh Reid

Respect Where Due

Spammers are not fools, they identify that as a growing number of email users start using these techniques their percentages get squeezed, and so they are using various techniques to erode the effectivness of the anti-spam tools. The first way they did this was to use real email addresses as the from header, then they made the content of the emails constantly variable, but now there are new eroding techniques designed to reduce the ability of a well trained bayesian filter. These techniques are as follows:

  • The random word email, designed to fill the filter with vast quantities of words and slow it down. It also has the effect of diluting the words of non-spam emails making false positives more likely.
  • The blank email, these either get classified as spam and so make the normal email header information be associated with spam and so trigger more false positives; or they get through and affirm the sender's address as not being spam.
  • The CC loaded email, these get classified as spam but they cause the domain name of the recipient to be strongly associated with spam. This form have the side effect for domain owners of delivering 10 copies of the same spam, a heavy load for slow spam filters.
  • The literary quote email, the subject and content look like normal text because it is normal text written by a human being. It has the same effects as the random word email, except that you will have great difficulty in using textual analysis to spot it automatically.
While these techniques may not appear to be very disabling to many filters, they are prelude to a whole series of increasingly sophisticated attacks on the filters and rules. The spammers objective is to get one phrase or url through a filter where all other spam has failed, or to make the filter too difficult to use effectively.

Spammers also employ techniques to disable the email client filters that come after the MTA filter has correctly identified a spam message. This takes the form of a strange email header that although technically can be delivered causes the email client to not parse beyond that header for the header set by the filter, and so not perform the mail processing rule associated with it. Sometimes these odd headers cause the filter itself or the pop reader to fail, leaving messages clogging up your inbox.

The other reason why the spammers will always win is that anti-spam is a good business. Just look at recent dealings in the better tool provider companies and you will see how good. None of the players in the industry have a good financial reason for really solving the problem. But why isn't there a 'this will stop spam' solution? 2 reasons:

  • The internet still preserves anonymity, legally or not. And so spammers will always be able to send email, even if there are no open relays.
  • The mail transfer protocol SMTP is not designed to prevent the sending of millions of spam emails.

Or try:
bayesian filter
black lists
honeypot addresses
mail rules
email client filters
MTA
SMTP
micropayments
identity signatures
sender mailboxes
anti-spam legistation
Related Pages

In Hope...

So what can be done? I should be possible to enhance/replace SMTP to do 2 things better, tracking and reactivity. But this is a major upheaval to both the fabric of the internet and the email client tools. Current ideas of exactly how to do this involve things like micropayments, quotas, identity signatures, sender mailboxes and direct mail delivery.

Microsoft are in the best position to lead the changes, and are starting that process: www.microsoft.com/spam/ . Governments are putting anti-spam legistation in place to deter spammers.

In conclusion, I think that nothing short of a total rethink of global email will slow the spammers; and as the internet community is never 'of one mind' then this will be very unlikely to happen. So no matter how good your filtering, the spammers will always win.

Copyright © Hugh Reid, Creative Commons License
This work is licensed under a Creative Commons License.