Spam is a serious problem on the Internet, and there is no perfect solution. But here's what you can do when using our service.

First, your domain must have the antispam service enabled. We don't charge extra for it, but we only enable it on request because it costs us something. If the images in this topic don't look right and you can't find any of these antispam features, have your email domain administrator drop us a note and ask that we enable antispam for your domain.

Once your domain has antispam enabled, all incoming mail will be scanned by the antispam engine, and marked by adding special headers. Delivery is not affected by the scanning engine, and all the spam will still land in your INBOX until you take the next step:

Each user must enable the Junk Mail Filter for his own account. This looks for a special header inserted by the scanning engine, and diverts marked mail into the user's Junk Mail mailbox. The user can then review the contents of this mailbox and delete messages that are not wanted.

By default, this Junk Mail Filter is OFF. You must act to enable it, or all the spam will still show up in your INBOX, special markings or not.

See also the announcement about more antispam technologies that we have brought to bear.

To enable your Junk Mail filter:

Log into Webmail. Select Options, then Message Filters. Edit the Junk Mail Filter (click on the pencil icon); check the box for Normal and hit OK, then Close.

Like this:

Will it catch all the spam and nothing else?

No scanning engine is perfect, and "unwanted" is highly subjective, so it is not generally a good idea to simply discard any message that it marks. A legitimate message might be marked because it resembles spam; this is a false positive. And of course not all spam is detected, so some of it will end up in your INBOX; these are false negatives.

False negatives are easy to handle: just delete them. But false positives are more of a problem. Either you need to review (if not read) all the messages in your Junk Mail folder every so often, or you can wait for some outside intelligence, perhaps a phone call, to alert you to a missing message. In the former case, it's often quite effective simply to scan the sender's names of the identified spam in your Junk Mail folder: humans are pretty good at spotting names of people they know in lists.

In the latter case ("see no evil"), we can arrange for auto-expiry of your Junk Mail, deleting all messages older than N days (where you pick N): Drop us a note to set this up. If the consequence of a false positive is not too bad (you're not expecting a million-dollar purchase order from a total stranger), this is a sensible plan, and relatively trouble-free. Pick a value of N that will let you go back as far as you need to in case of a real false-positive, and it will be there in the file. Values of 3, 7, and 14 days are popular.

What about mailing lists?

Some very large mailing lists send out self-similar mail at rates that will trigger a high score and be marked as spam. If you subscribe to such a list, add another filter ahead of your Junk Mail Filter that detects these and files them directly in INBOX, then stops. The Junk Mail Filter will never run on those messages, so they will get through.

Like this:

By default new filters are added at the bottom. Hit the up-triangle next to this one (or the down-triangle on the other) to re-order the filter list so the one for this mailing-list runs before the Junk Mail Filter:

(is sub-addressed mail filtered?)

Can I block or exempt certain addresses?

Yes, each user has a blacklist (which blocks) and a whitelist (which exempts). These can contain sender email addresses, entire or with just a domain name in which case it applies to all addresses in that domain. (The term "sender" here means an address in the From: header, not -- alas! -- the SMTP envelope sender, the Return-Path: header, or the Sender: header, if any. There are filter primitives to look at these things, but no operator that compares against just the blacklist.)

There are also per-domain black- and white-lists manageable by the email domain administrator, and a feature called Mailing List Exemptions, that will exempt certain recipients in the local domain from antispam processing.

Your Junk Mail Filter also has another setting, "exclusive", which makes it pay attention only to the whitelist, ignoring any score: Any sender address not on your whitelist will trigger the filter. But you will have a job to clean out the Junk Mail and manage your whitelist. It may make sense for some users. (You can run two filters in series, "exclusive" then "normal", and deliver things to two separate mailboxes, if this helps you. One of us did it this way a while back, but the newer-technology scanning engine made it no longer necessary.)

Where do I report false positives (or negatives)?

The system we use does not depend on user reports, so you needn't bother. Just delete the false negatives, and maybe whitelist the false positives. The system doesn't learn by listening to the end-users, rather it listens for "storms". See Technology, below.

From within Webmail, when antispam is enabled for your domain, there are links "This is spam" (when reading the Junk Mail folder) and "This is not spam". These each do two things: they move a message from INBOX to Junk Mail (or the other way), and they add the sender's address to your black- or white-list, respectively.

Will this work with my client's adaptive antispam filter?

Not automatically. There are systems that learn from your feedback what you think is spam, and many of these run in the context of the client, because (a) that's where the buttons are, and (b) they are not scaleable to run at the server level for thousands of different users. You can certainly give anything a try, but there is no standard way for the client-side antispam solutions to feed things back to the server-resident filters. Not yet, anyway. If you want to add something to a server whitelist, you must log into the server to do it. (If you write such systems, talk to us about having the client use our administration protocol to manage these lists under the covers. Alone among email providers, we give users access to this protocol, for just this sort of thing.)

Technology

Our Mirapoint message server has their Signature Edition antispam system integrated, which is based on Commtouch technology. This system does not look for patterns in the usual sense, as e.g. SpamAssassin does, so there are no "bad words", hence no problems for people who legitimately need to communicate about mortgages, say, and no problems detecting non-English spam.

Instead, it generates mathematical hashes or "signatures" for pieces of each message, and reports them to a central clearinghouse. (The messages themselves are still private, and no identifying information is sent.) If the clearinghouse is detecting a high-volume attack of mathematically similar messages being delivered on the Internet right then, it will return a high score.

We use this ourselves of course, for instance in front of the support intake address, and we're generally very happy with it. It's among the best systems out there right now.

The headers that are inserted in each message look something like this:

X-Junkmail-Status: score=10/50, host=m1.imap-partners.net
X-Junkmail-SD-Raw: score=unknown,
    refid=str=0001.0A090205.44E2E92D.0058,ss=1,fgs=0,
    ip=1.2.3.4,
    so=2006-03-30 10:46:40,
    dmn=5.2.113/2006-07-26

That's a regular message with a score of 10, the minimum, and not (identified) spam. A score of 50 is the level where it's deemed to be probable spam, and a score of 300 (the maximum) is definitely spam, or anyway "bulk". (The rest of it -- like email headers generally -- is meaningful only to the right technical person, when diagnosing a problem.)

Your Junk Mail filter, when in the "normal" setting, looks at headers like this:

X-Junkmail: UCE(300)
X-Junkmail-Status: score=300/50, host=m1.imap-partners.net
X-Junkmail-SD-Raw: score=confirmed,
    refid=str=0001.0A090206.44DE0C4B.0020,ss=3,pt=4368,fgs=0,
    ip=62.140.23.56,
    so=2006-03-30 10:46:40,
    dmn=5.2.4/2006-05-04

and when it sees that "X-Junkmail" header containing the string "UCE", it acts, doing whatever you have told it to do; typically filing the message into "Junk Mail", but it could be any action you like.

You can also construct other filters using the ":ucescore" primitive and the numeric operators, such as "greater-than 299", and make your own system of buckets, but most people don't have that kind of spare time. Some of our customers make domain-level filters that discard above a score of 299 (there is no >= operator), though this may not work for users in all domains.

Edited 4 time(s). Last edit at 12/27/2006 11:36AM by jxh.

Can I set auto-expiry values myself through the command line interface for Junk Mail and perhaps Trash folders, or do I have to ask you to do that for me?

mlb

We have to do it.

A question or two about spam filtering.

Right now I am configured (at the domain level) to scan the incoming email. Then, at the user level, I have hand-created filters that file messages into a junk folder based on the UCEScore parameter.

(1) Is there a way to list certain source addresses or subject line strings for which mail should not be scanned? (Through a whitelist or an addressbook entry, for example?) Or do I just have to set up filters to catch those which precede the UCEScore filter?

(2) I know the webmail interface includes a way to tag a message as spam. Is there any way to tell the system about a false positive? Also, I normally use the Mulberry IMAP client--is there any way to mark messages as spam other than through the web interface?

Thanks!

Dave Scocca

(1) Is there a way to list certain source addresses or subject line strings for which mail should not be scanned?

The ":uce" filter primitive pays attention to the white- and black-list (for the user or domain, depending on the scope of the filter), but the ":ucescore" primitive does not. The "normal" setting also looks at the score, against a default threshold of 50, but the "exclusive" setting looks only at the whitelist. I would suggest you use both filters in series, in some combination, to get the effect you want. For example, ":uce is exclusive" first, filing to the INBOX if it triggers, then falling through for other processing by e.g. the ":ucescore is" primitive.

(2) I know the webmail interface includes a way to tag a message as spam.

Well, the "This is spam" button mostly just moves the message to your Junk Mail folder, and adds the address to your blacklist. (The "This is not spam" button, when viewing a false-positive in the Junk Mail folder, moves it to the INBOX and adds it to the whitelist.) It can also file a copy in a system folder that might be fed back to the antispam provider, but we're using the form of the engine that does not require such feedback for its operation, so this is basically a no-op.

Also, I normally use the Mulberry IMAP client--is there any way to mark messages as spam other than through the web interface?

Well, any IMAP client can of course move the message to the Junk Mail folder. To get it to also add a whitelist entry, you need to either use Webmail, or explicitly add this with the CLI or the protocol, using the UCE ADDEXCEPTION command. In principle, this could be scripted, but I don't think anyone has yet integrated such a thing into Mulberry. Personally I just use SSH and paste the address into the UCE ADDEXCEPTION command when I feel strongly about it, or I make a filter (that runs ahead of the ":uce" filter) that detects e.g. the Return-Path for a given mailing list I'm on.

We've been saying that false positives are seldom a problem with this engine. Do you find that it catches things sent to large mailing lists you're on, or some other reason? I know we've seen false positives when customers send us examples of spam they got -- which then matches the signature enough to get a high score -- but we very seldom hear of any other kind.

It took me a while, but I finally got around to working with this.

I got rid of all my old hand-written filters and replaced them with two using the rules. The first rule files to my inbox using:

:UCE is-not exclusive

("is exclusive" returns FALSE for addresses on the whitelist and TRUE for other messages, so "is-not exclusive" is necessary here)

The second rule files to my junk folder using:

:UCE is normal

So far I've had only one or two false positives, both of which were corporate mass mailings, so I added them to my whitelist and all was well.

A final question: I've also had two or three missed spam. Is there any reporting mechanism by which these can (or should) be fed back into the engine?

Thanks!

A final question: I've also had two or three missed spam. Is there any reporting mechanism by which these can (or should) be fed back into the engine?

Not at this time. This engine does not rely on end-user feedback; it has other methods.