Filtering Mail for Spam


All mail arriving at Steward from outside the department will be passed through a set
of filters which attempt to classify it as spam or non-spam (also known as "ham").

Once the classification has been made, special headers are inserted into the mail message, and
the mail is delivered as usual. Note that all mail will continue to be delivered.
You are responsible for filtering the incoming mail and doing with it what you will.
The computer support group will not make decisions about the content of your mail
or what to do with it other than to add these spam classification headers.

The only exception to this rule is when mail is found to be infected by a known virus. In
this case, the virus is removed from the mail (or attachments) and the mail is delivered
along with a notification of what was removed.

For the forseeable future, we do not recommend simply throwing away messages filed
as spam. We expect to be trying new filters from time to time, and while we test these
filters extensively before we add them to the system, we cannot be sure that they
always lead to the desired result. Check your spam regularly, just in case...

In our initial attempt at filtering, four headers will be added to each message.
For example, a recent spam message produced:

    X-MailScanner: Found to be clean
    X-MailScanner-Information: Please contact the ISP for more information
    X-MailScanner-SpamCheck: spam, SpamAssassin (score=7.2, required 5, SUBJ_HAS_Q_MARK,
          PLING, GUARANTEE, FULL_REFUND, DOUBLE_CAPSWORD, CLICK_BELOW)
    X-so-MailScanner-SpamScore:
sssssss

The first header indicates that the message was not found to be infected by a virus.

The third header gives an overall score for "spamness". Generally, a score of 5 or greater
indicates a high probability of being spam. It then gives some indication of why the message
was classified as spam.

The last header is the one to use for filtering spam from your email. It gives the score as a string
of "s"-es. We recommend you set up your mail filtering to search for a string of at least five
"s"-es in this header and then take some action, such as moving the message to a "PutativeSpam"
folder. Instructions on how to do this with various mail-reading programs are given below. You
may wish to alter your "spam threshold" as you gain experience with filtering. At present, the
filters we employ seem to remove about 85% of spam. We have a few more filters yet to
implement, and hope to push this number into the 90% range soon.

In addition, mail with a score of more than five "s"-es will have "{Spam?}" inserted at
the beginning of the subject line. In this case, the mail is delivered with the subject:

    {Spam?} Eat pizza, watch TV ... AND LOSE 22 POUNDS!
From:
<brocktony@msn.com>

Instructions for Various Mail Readers

In the following, you will be instructed to move all spam to another mail folder. Most mail readers
provide an option for throwing away mail. We strongly recommend that you do not throw away
your spam, at least not for a few months until you are confident that the classifier is doing what
you want. Periodically check the mail classified as spam, and notify the computer support group
if you find mail which has been erroneously classified as spam. Once you have done this, you can
bulk delete the entire contents of the spam folder, confident it will soon fill up again.

Here are instructions for setting up your mail reader for filtering spam based on
the special headers introduced above.

Webmail:

(When webmail is introduced later this month, instructions on filtering will be provided here)

Unix Mail Readers:

Mozilla, Netscape 7, and Netscape 4.7.8:

  1.  Choose Message Filters from the Tools menu (for Netscape 4, choose from the Edit menu):
  2.  Click the New button on the "Message Filters" window which appears
  3. Give the filter a name (such as Spam Filter) in the "Filter Rules" window which appears.
  4. Click the down arrow next to Subject, and choose Customize.
  5. Type in the name of the header to check, X-so-MailScanner-SpamScore (without the colon)
  6. Click Add, and OK
  7. Click the down arrow next to Subjectagain, and select the new header you just added
  8. In the empty box following contains, type in five "s"-es: sssss
  9. Below, select "New folder"
  10. Type in the name of the mail folder you want your spam moved to (e.g. PutativeSpam).
  11. Click on Click here to select
  12. Move to your email address, and select choose this for the parent
  13. Click OK
  14. Click OK in the "Filter Rules" window
  15. Close the "Message Filters" window.
All mail recieved with a spam score of five or above will now be automatically moved to
the folder you created (e.g. PutativeSpam). You can use this filter to do other mail manipulations
as well, such as moving all mail from a given address to a certain folder, etc.


Pine:

(Press the boldfaced characters to generate actions)
  1. If you are not there, go to the Main screen
  2. From there, go to Setup->Rules->Filter
  3. Add a rule.
  4. Initially, the "Nickname" field should be highlighted.
    Add a value at "Nickname", e.g. Spam Filter.
  5. Select (*) the "Specific" choice under "Current Folder Type"
  6. Use Change, Add, or Delete to make "Folder List" read INBOX.
  7. Press X to add a new header to filter on, entering X-so-MailScanner-SpamScore
  8. The "X-so-MailScanner-SpamScore" header should now be highlighted under
    "FILTERED MESSAGE CONDITIONS BEGIN HERE" Use Add value,
    C    hange value, or Delete value to make the value field contain five "s"-es: sssss
  9. Scroll down to the "ACTIONS BEGIN HERE" section.
    Make sure the "Filter Action" option is set to "Move".
  10. Select the "Folder List" item in that section, and set it to the name of the
    mail folder you want your spam moved to (e.g. PutativeSpam).
  11. Exit setup and press Y to commit the change.
    You may be asked to create the PutativeSpam folder; accept that change if this happens.
  12. Exit setup again, and press Y to commit the change.

Sun MailTool, Elm, and plain old Mail:

The Sun MailTool, Elm, and plain old UNIX Mail cannot do mail filtering. We suggest that you
switch to using Netscape/Mozilla or webmail as your mail reader. If you must continue to use
MailTool or Mail (why we cannot imagine), see Alan, Jeff, or Neal for instructions on how to set
up a work-around mail filtering script using procmail.


Windows Mail Readers:

Eudora:

  1. From the main Eudora window, select the drop down menu Special, then select Make Filter.
  2. Click Add Details.
  3. Click New to make a new filter. Notice the Untitled name in the left of your screen.
    Now check the box next to Incoming under the Match category.
  4. In the Header drop down menu, select <Any Headers>.
  5. The next drop down menu should read contains. In the empty box next to it,
    enter X-so-MailScanner-SpamScore
  6. You can safely leave the next menu as ignore. (This part is where you can daisy-chain
    filters one after another.)
  7. Under the Action section, the first drop down menu should read Transfer To,
    then click the button immediately across from it.
  8. When you click the button, it will present you with a list of existing mailboxes
    and a New.. selection.
  9. Click New and a window will pop up asking you to name a new mailbox.
  10. Enter the name of the folder to which you would like mail classified as spam to be moved
    (for example PutativeSpam)
  11. Click OK.
  12. Close the "Filters" window, and click Yes to confirm.


Outlook:

Note that this is not Outlook Express (see below). Note also that Outlook has had so many
security vulnerabilities in the past that we cannot recommend its use; Outlook Express is
safer.
  1. Go to Tools->Rules Wizard...
  2. Click 'New...' (on the top right)
  3. Choose 'Check messages when they arrive'
  4. Click 'Next'.
  5. Check 'With specific words in the message header'.
  6. Click on 'specific words'.
  7. Type in: X-so-MailScanner-SpamScore: sssss (one space between the : and the first s)
  8. Click 'Ok'.
  9. Click 'Next'.
  10. Check 'Move it to the specified folder'.
  11. Click on 'specified'.
  12. Create a new folder to which you would like mail classified as spam to be moved (e.g.  PutativeSpam)
  13. Click 'Ok'.
  14. Click 'Next'.
  15. Click 'Next'. (Again, unless you want to add exceptions.)
  16. Give the rule a name. (The default is what you typed for specific words above.)
  17. Check 'Turn on this rule'. (You may or may not want to check 'Run this rule on my Inbox now'.)
  18. Click 'Finish'.


Outlook Express:

Outlook Express cannot filter on mail headers. Thus, you will not be able to customize
your mail filtering with this tool. If you are content to use the computer support group's
spam cutoff score of five, you can create a filter based on the "{Spam?}" entry in an
incoming mail's Subject header as follows:
  1. With Outlook Express open, move to the Tools menu and select the
    Message Rules     option, then Mail
  2. If the "New Mail Rules" window does not appear, click on the New button in the upper right of the "Message Rules" window;
    the "New Mail Rules" window is displayed.
  3. Respond to 1.) Select the Conditions for your rule by checking the box next
    to "Where Subject line contains specific words."
  4. Respond to 2.) Select the Actions for your rule by checking the box adjacent to "Move it to the specified folder".
  5. Respond to 3.) Rule Description (Click on an underlined value to edit it) by clicking
    the highlighted words "contains specific words"; the "Type Specific Words" window will appear.
  6. Type {Spam?} and click the Add button and the OK button
  7. Click on the highlighted word "specified"; the "Move" window will appear.
  8. Choose a folder to which you would like mail classified as spam to be moved, or create a new one, then click OK.
  9. Respond to 4.) Name of the Rule by highlighting the words "New Mail Rule #1" and typing Spam Filter.
    Click the OK button.

Mac OS X Mail Readers:

  1. In the menu bar click Mailbox thenNew Mailbox and create the mailbox to
    which you would like mail classified as spam to be moved (e.g. PutativeSpam)
  2. In the menu bar click Mail then Preferences...
  3. Click Rules then Create Rule
  4. Add a description of the rule, then click the From Criteria then click Expert...
  5. In the "Header:" field enter X-so-MailScanner-SpamScore, click Add Header and OK
  6. Now click From and select X-so-MailScanner-SpamScore
  7. Select Contains in the next box and enter yes in the third "Critera" box.
  8. In the "Action" section, check 'Transfer to mailbox' and select the desired mailbox. click OK
  9. Adjust the rule priorities if you want and dismiss the "Mail Preferences" dialog box.