My friend Darrin Chandler wrote regarding my June 16 entry about spam filtering.  He pointed me at SpamAssassin and Vipul’s Razor, two more attempts at spam filtering.  And, sadly, two more that are bound to fail.   SpamAssassin plugs into a number of different places in the email flow, and uses heuristics and blacklists (like Vipul’s Razor) to attempt to identify and filter the spam.  SpamAssassin is for Unix and Unix-like (i.e. Linux, BSD, etc.) operating systems.  Perhaps a Windows port is forthcoming?  Vipul’s Razor is, according to the web site:

…a distributed, collaborative, spam detection and filtering network. Through user contribution, Razor establishes a distributed and constantly updating catalogue of spam in propagation that is consulted by email clients to filter out known spam. Detection is done with statistical and randomized signatures that efficiently spot mutating spam content.

We’ll see how well it works.

Darrin also pointed out a fundamental flaw in my trusted server idea:  getting a critical mass of important ISPs to subscribe to such a thing.  If AOL were to do it, for example, they’d get millions of emails from irate subscribers who all of a sudden can’t get email from Aunt Tilly, because Aunt Tilly’s ISP doesn’t subscribe to the trusted server network.  That’s really too bad.  It leads me to believe that most people don’t really care about spam.  Sure, they’d like not to get it, but not if it means having to change ISPs or possibly lose communication with Uncle Harold.

A friend of mine ran a spam filtering idea past me last week:  Rather than accept everything and then spend time filtering the trash from your inbox, why not create a list of people from whom you’ll accept mail, and then block everybody else?  The system he envisioned would still accept mail from non-approved senders, but put it in a separate folder for you to scan at your leisure.  This has the benefit of ensuring that what ends up in your inbox is all good.  He had a blocked senders list, of course, and also word filtering. Monday, Jeff Duntemann mentioned MailCircuit, which implements this “opt-in” type of mail filtering, in his web diary.  Perhaps some people will find this kind of filtering useful.  It seems like more work to me, though, or at least the same amount of work.  The only thing it does is unclutter my inbox, but I still have to scan those other messages.  I’m not interested in any anti-spam measure that requires more work on my part.

On Tuesday, osopinion.com ran this article by John Christie advocating what he calls “Reverse FIltering.”  His system, too, is an opt-in list.  You set up a list of approved senders whose mail will always be accepted.  Messages from non-approved senders will be bounced back, with a message describing how to get onto the approved list.  That method typically would be to visit a web page and fill out an application of some sort—presumably an application that cannot easily be filled out by an automated system.  This certainly would work, but it would block way too much legitimate mail.  For example, if you give your email address to anybody, you’d better be sure to add those people to your approved senders list before they send a message.  This includes friends and relatives, as well as web site signups that require email validation.

Others in the Talk Back Forum for the OS Opinion article have put forth variations on these opt-in systems.  All of them suffer from one or more of the problems that I’ve already pointed out.  Perhaps the most common mistake is relying on the “from” email address for filtering.  That value just isn’t reliable—it’s ridiculously easy to put a bogus address there.  The “Reply-to” and “Return-Path” addresses also are unreliable.  It’s impossible to write a reliable filter based on those addresses.

I’ll say it again.  The only reliable way to prevent spam from getting to your inbox is to prevent it from entering the SMTP mail system.  Period.  Once it’s in the system, it’s impossible to filter without also filtering some potentially legitimate messages.  I have yet to see (and I’ve been looking!) any proposal for filtering spam that would be as unobtrusive or as effective as the system of trusted servers that I suggested in my May 15, 2001 entry.  Is nobody giving this problem any serious thought?  Are they all sitting on their hands waiting for legislators to come up with a solution, so they can then complain about how the new law is ineffective and overly burdensome?  I’m very disappointed.