MozillaZine

Message Filters

From MozillaZine Knowledge Base

Message filters are useful if you routinely want to perform certain actions on messages. For instance, you can have incoming mail automatically sorted into different folders and certain messages tagged, marked as Junk, forwarded or even deleted. Filters can be applied automatically to incoming mail, or you can run them manually when desired.

Contents

Creating and editing filters

This tutorial has step-by-step instructions (with screenshots) for how to set up message filters in Thunderbird. You can also create a new filter from any message by right-clicking on the email address of the sender in a message's headers, and choosing "Create Filter From Message...."

Filters are account specific, there is no support for global filters. The closest thing to a global filter would be to configure several POP accounts to use a Global Inbox. In that case, you could create filters for the Local Folders pseudo account that would be applied automatically to incoming mail for those accounts.

The file that stores your filter settings is called "msgFilterRules.dat". Each account will have its own "msgFilterRules.dat" file, stored in the "Local Directory" for the account in your profile. The Local Directory is specified at the bottom of Tools -> Account Settings -> Server Settings, right by the Browse button. The filters for Local Folders are stored in the Mail\Local Folders directory in the profile.

If you want to duplicate an accounts filters in another account you don't need to create them again. Exit Thunderbird and copy the "msgFilterRules.dat" file instead. If the account is in another profile and it uses custom headers you will also need to copy the mailnews.customHeaders setting from prefs.js.

There is no support for sorting rules in the message filter window. If you have problems finding a filter try using the Filter Of Filters add-on. However, if you have hundreds of rules it is usually easier to edit them by editing the "msgFilterRules.dat" file with a text editor (not a word processor).

Running filters

Enabled filters are run automatically, in the order they are listed in the "Message Filters" dialog ("Tools -> Message Filters..."). If a filter moves a message to another folder subsequent filters will not be automatically applied to that message once it has been moved from the Inbox. There is no built-in support for automatically running filters on outgoing mail. Install the Send Filter add-on if you want to do that.

To run filters manually on the currently opened folder, select "Tools -> Run Filters on Folder". To manually run only some of your filters rather than all of them, go to "Tools -> Message Filters...", select the filters you want to run, and click the "Run Now" button.

When you create a filter "Apply filter where" defaults to "Checking Mail or Manually Run". That means the filter will be run automatically when Thunderbird checks for new mail in the inbox for that account, and if you manually run the filter using Tools -> Run Filters on Folder or Tools -> Run Filters on Message. The list box can be used to fine tune exactly when the filter is run. For example, "Checking Mail (after classification)".

Filtering the message body

You can filter based on the contents of the message body if you select "Body". If you have a message with both a plain text and a HTML version of the message body it searches both copies. This works for both local and remote (IMAP) folders. You used to be able to search the entire message (including all headers and the attachments) if you selected "Body" though in some releases you had to add it as a custom header (see below) to use it with an IMAP account. Nowadays Body is only predefined for POP accounts, and it only searches the message body. If you want to filter the message body of a IMAP account you need to add a Body custom header (see below).

The official reason is that Thunderbird only downloads the message headers before applying filters, and adding support for filtering the message body in an IMAP account would require adding support for reading the offline folders (offline message store) in the IMAP filter code. [1]

Custom headers

Message filters don't know about every type of message header. For example, any header beginning with a X such as X-Spam-score:. If you want to use a unknown header select "Customize" at the bottom of the leftmost list box (while editing the filter rule). Then either select a custom header from the list box or add one by entering its name in "New message header:". Omit the ':'. For example, if you have a Message-ID: <mailman.73.1328558416.25839.tb-enterprise@mozilla.org> header and want to test Message-ID: headers you would enter Message-ID (omit the trailing ':').

This creates a custom header that is stored using the mailnews.customHeaders setting. You can use it with other filters. It will show up in the normal list of headers, you don't need to use Customize again.

Spam

While you can create message filters to test for Spam thats re-inventing the wheel. Its more efficient to use the junk mail controls . This article explains how to tweak the the junk mail controls to make them learn faster. You could also run other tools such as SpamAssassin or SpamPal that add headers to describe its analysis of the message and some sort of spam score that you can test. For example, SpamPal adds a X-SpamPal: PASS header if the message isn't spam. You could test that yourself, or configure the junk mail controls to trust SpamAssassin or SpamPal and have it test it instead.

When you check for new mail message filters are run before the junk mail controls. There is no way to change that order.

Testing whether a message is marked as junk

The message filters don't support testing whether a message is marked as junk. The information seems to be stored in the index file, not one of the two MozillaStatus headers. However, if you configured the junk mail controls to trust either SpamAssasson or SpamPal you could look in the SpamAssassin.sfd or SpamPal.sfd file in the isp directory in your Thunderbird program directory and figure out what it tested in order to mark the message as junk, and test the same thing. For example, the following line in SpamAssassin.sfd

condition="OR (\"X-Spam-Status\",begins with,Yes) OR (\"X-Spam-Flag\",begins with,YES) OR (subject,begins with,***SPAM***)"

means that SpamAssassin normally tests whether the X-Spam-Status or X-Spam-Flag header begins with Yes or the subject begins with ***SPAM***.

Wild cards

Thunderbird doesn't have built-in support for using wild cards or regular expressions in filters. The FiltaQuilla add-on supports regular expressions. If you look on the authors web page it describes how to use either javascript or regular expressions in the search field.

However, sometimes you can get by using multiple rules instead. For example, to test for From: addresses that look like no-reply463@workingin-australia.com, no-reply321@workingin-australia.com, and no-reply789@workingin-australia.com you could create a single filter with two rules, and check "Match all of the following". The first rule would be "From" "begins with" "no-reply" and the second rule "From" "ends with " "workingin-australia.com".

Account specific filters

Each account has its own set of message filters, but the filters have no concept of accounts. You can create account specific message filters by checking the value of the X-Account-Key header.

Use View -> Message Source and find the header, noting what its set to. For example, X-Account-Key: account10. Then create a message filter and scroll the leftmost list box (it defaults to Subject) down to Customize. Enter X-Account-key as the new message header, and press OK. Select X-Account-Key in that list box and test whether "X-Account-Key" "is" "account10" (replace account10 with the value your account uses) and select in "Perform these actions" what it should do.

If the filter doesn't work see if Thunderbird changed the header it tests to Subject behind your back. Sometimes you run into a temporary glitch after adding a customized header. If that happens, exit Thunderbird, restart it , and edit the message filter to use X-Account-Key.

Troubleshooting

  • Check whether the filter is enabled. It should have a check mark next to it in the enabled column.
  • If all of the message filters for an account disappear when you start Thunderbird, exit Thunderbird and delete any "tmprules.dat" files in the profile. They are in the account directories, the same location where the message filters ("msgFilterRules.dat") are normally stored. [2]
  • A bug prevents tags from being automatically applied by filters if you enable "allow antivirus clients to quarantine individual messages". It will work if you run the filter manually.
  • Try running a filter that always matches, and sets something that you can see. That will help identify if you accidentally edited the filter for the wrong account.
  • Try running the filter manually. There might be nothing wrong with your filter, and a bug prevented it from running automatically.
  • Use a text editor to look at the "msgFilterRules.dat" for that account and see if it stored what you thought you specified.
  • Its frequently safer to test using "contains" rather than "is", especially if you had signed up for a newsletter. They might make a trivial change in the From address or subject that breaks your filter. Use "View -> Message Source" to look at the message source to eliminate any possibility that what you see when you read the message is not what the message filter saw.

If you have never written a message filter before and can't figure out what is wrong, try starting smaller and working your way up. For example, you might have written a filter to move certain new messages to a different folder and it doesn't seem to do anything. Modify the message filter to make it add a star to all new messages. That is something harmless you can see and easily undo manually. If that works then modify it to add the star only for those messages that you want to move to a folder. If that works then modify it to move the messages instead of adding a star. The reason for doing this is by starting small its easier to catch and debug common errors, or identify if there is something wrong with that account that prevents any message filter from working etc.

You could create a message filter that adds a star to all messages by testing whether the "Subject" "isn't" "~!@#$%^&*()_+i" (or some other extremely unlikely string) and selecting "Add Star" in the actions list box. Test it using "Tools -> Run Filters on Folder". If no matter what you do it doesn't work, try replacing the contents of the "msgFilterRules.dat" file for your account in the profile as follows. If that doesn't work you probably choose the wrong directory.

version="8"
logging="yes"
name="Apply a star to all messages"
enabled="yes"
type="1"
action="Mark flagged"
condition="OR (subject,isn't,\"~!@#$%^&*()_+i\")"

Alternatives

The Mail Classifier extension attempts to classify and move messages to other folders based on learning what messages you move, and to where.

If you have a IMAP account your email provider may support server side filters. That is a combination of message filters and junk mail filters that are run automatically by the server whenever it receives a message for your mailbox. Some may also provide the option to create custom scripts using the Sieve mail filtering language. There is a Sieve extension that lets you manage the Sieve scripts without having to use a browser. Unfortunately server side filters aren't available in free IMAP accounts provided by Gmail, GMX, or AIM.

Some alternatives to the junk mail controls provide powerful filtering capabilities but they typically don't support the commands needed to manage messages.

Other information

  • The Message Filter Import/Export extension can be used to import/export message filters. However, its only useful with Thunderbird, Netscape, Mozilla Suite, and SeaMonkey message filters since it doesn't know how to convert other formats. This extension will replace any existing filters when the import is run. That is, it does not merge two sets of filters.
  • To enable and view your filtered messages log, follow "Tools -> Message Filters," select "Filter Log," and check the box for "Enable the Filter Log." If you can't view your filter log for some reason look in the "filterlog.html" file in that accounts subfolder in the profile.
  • Eudora OSE added support for importing filters from the original Eudora plus some other message filter enhancements such as Tools -> Run Filters on Selected Messages.
  • Thunderbird 3.1.4 added a "Match All Messages" radio button that you can use to have one or more actions always occur. If you're using an older version you could test whether the subject doesn't match a string that you're unlikely to ever see such as %#!*&;2Q~! instead. i.e. "Subject" "isn't" "%#!*&;2Q~!"
  • "Size" in a message filter is in kilobytes. If you append a unit such as "KB" to that number it will silently save the setting as zero.
  • Message filters have different types. The types are defined in this source file.

See also

External links

  • The NewMail Attention add-on adds support for a message filter action to call attention to the main window when a new message arrives.
  • The SendFilter extension runs message filters automatically on outgoing messages after sending them.
  • The EagleEye add-on let you filter messages based on various statistics. It has been disabled on the addons.mozilla.org site. Its still available though at SourceForge.
  • The Sieve add-on lets you remotely manage Sieve scripts (server based equivalent of message filters) on your mail server.

Thunderbird 3 or later only

  • The Bulk List add-on is an experimental add-on to explore providing a set of search folders for popular services.
  • The Expression Search / GMailUI add-on adds more search expressions (both to the Quick Filter toolbar and the message filters)
  • The FiltaQuilla add-on adds support for about a dozen actions including launching or running a file and training the bayesian filter with the tokens in the mail message.
  • The QuickFilters add-on makes it easy to generate mail filters on the fly, by dragging and dropping mails and analyzing their attributes.
  • The Filter Sub-folders add-on lets you apply mail filters to sub folders of an IMAP account, not just the Inbox.
  • The Filter of Filters add-on adds a search bar that is useful if you have many filters.
  • The Mailing List Manager add-on provides automatic search folders and an option to filter into a real folder for messages sent to mailing lists.
  • The New Mail Attention add-on adds a 'Get Attention' action.
  • The RedirectFilter add-on provides the ability to forward a message using the writer's From: address.
  • The ToneQuilla add-on adds support for playing a specific sound based on the message class.
  • The Quick Filter Bar Pivot add-on extends the functionality of the Quick Filter Bar in Thunderbird 3.1 and later. The Quick Filter Bar evolved from the Quick Filter Prototype add-on, which requires Thunderbird 3.0.x. See this blog post for more information.

Folder classification

  • The MailClassifier extension uses Bayesian filtering to decide what folder to move a new message to.
  • The Nostalgy add-on supports rules to suggest a target folder for saving messages

Bug reports