MozillaZine

Compacting folders

From MozillaZine Knowledge Base

This article was written for Thunderbird but also applies to Mozilla Suite / SeaMonkey (though some menu sequences may differ).

When you delete messages in an email client such as Thunderbird they aren't physically deleted. Even emptying the Trash does not get rid of them. Instead they are marked for deletion and hidden from view. They are not physically removed until you "compact" the folder. This is a tradeoff done to improve performance in large folders.

"Compacting" a folder has nothing to do with compressing or zipping a file. Compressing/zipping only changes how data is stored; compacting changes the actual data. Compacting a folder frees wasted space but the main reason to do it regularly is because its a type of preventative maintenance. If you don't compact your folders regularly performance can slow to a crawl, you can't store new messages anymore, start losing parts of messages or have deleted messages resurrected. If you notice a message has a date of 1969 that is due to it losing the Date: header in the message.

Corrupt folders tend to occur when you have lots of messages in a folder, many of them are deleted, and you don't compact often enough. This is usually only a problem for the inbox folder as while other folders may get very large you typically don't delete messages in them often. Its recommended that you keep the number of messages in your inbox small, and move any you want to permanently keep to other folders/child folders.

Contents

How to compact folders

Its recommended that you configure Thunderbird to automatically compact folders rather than try to guess how often is often enough. Automatic is the default starting in version 5. However, a few people have problems due to automatic compacting while something else is happening, if this occurs switch to working offline using File -> Offline -> Work Offline, select an account and then manually compact the folders in that account using File -> Compact Folders instead. You can also compact a specific folder by right clicking on it and selecting "Compact" from the context menu.

Automatically

The easiest way to compact is to let Thunderbird do it automatically. If you are running version 5 or newer Tools -> Options -> Advanced -> Network & Disk Space -> "Compact folders when it will save over X MB" (Thunderbird) or Edit -> Preferences -> Mail & Newsgroups -> Network & Storage -> "Compact all folders when it will save over X MB in total" (SeaMonkey) is automatically enabled and defaults to 20MB. If you are running versions prior to Thunderbird 5, check the checkbox for "Tools -> Options -> Advanced -> Network & Disk Space -> Disk Space -> Compact folder when it will save over 100 kB" and then press the OK button.

Some people prefer to use a large number to reduce how often they run into a delay due to Thunderbird compacting folders. Others prefer to use a smaller number since it takes far less time to compact when you compact often. Regardless of which approach you choose keep in mind that its not just a question of wasted space, you want to avoid having too large a percentage of deleted messages in the folder to help reduce the chance of it becoming corrupt. 1MB is usually recommended, and in recent versions it is the minimum possible value.

This limit used to be stored in mail.purge_threshhold, in KB. Thunderbird 5 and later versions (and SeaMonkey 2.2 and later) ignore that setting and use mail.purge_threshhold_mb in MB instead. (see bug 437657).

mail.purge.ask defaults to false in recent versions of Thunderbird and of SeaMonkey, to disable being prompted if its okay to automatically compact the folder. In old versions you may need to modify that using the Preferences or Options dialog (see above), Thunderbird's config editor or SeaMonkey's about:config.

IMAP

You have several additional options if you are using a IMAP account. If you check "Tools -> Account Settings -> Server Settings -> Clean up ("Expunge") Inbox on Exit it will compact the Inbox folder when you exit Thunderbird. If you check "Empty Trash on Exit" it will also empty and compact the Trash folder when you exit.

If you don't like the performance tradeoffs due to automatically compacting when a certain amount of space is wasted you can configure Thunderbird 3 and later versions to auto-expunge messages instead. This will physically remove deleted messages when you have 20 deleted messages. You can also replace 20 with a different limit. See Deleting messages in IMAP accounts if you want to do that. If you're using offline folders and configured to AutoSync (Thunderbird 3.0 and later does this by default) it will expunge the deleted message in both the remote folder and the offline copy of the folder (an mbox file).

Add-ons

The web page for some of these add-ons state they add a button to the toolbar. However, it really just adds the button to the set of buttons available for the toolbar. The button is not displayed unless you customize the toolbar to actually add the button. You can customize the toolbar by right clicking on it, selecting customize, drag and drop the button to the toolbar, and then press OK.

The Xpunge extension lets you add a button to empty the trash and compact the folders in multiple accounts (in one step). You can also set a timer to automate that.

The CompactFolder add-on lets you add a button to the toolbar to compact the currently selected folder.

The MailTweak add-on has a "Compact from folder properties" tweak. It adds a Compact button and shows the unused space in the folder file as a percentage of its size on disk in the folders properties. It also disables Compact in the context menu if the folder is already compacted.

The Mozilla Add-ons web site has several other extensions that add buttons to compact folders.

Undoing compacting

You cannot undo compacting, there is no such thing as "uncompacting". Compacting physically removes messages marked for deletion and hidden from view. Once they're gone, they're gone. Usually when somebody asks how to "uncompact" a folder they lost messages due to compacting. Sometimes the messages were already gone, but that wasn't noticeable because they were still listed in the folder listing. Other times compacting a folder physically removed messages it shouldn't have because the folder was too corrupt.

If you suspect a folder is badly corrupted, don't compact it. It might make things worse because compacting depends upon Thunderbird being able to parse the mbox file (the text file named after the folder that contains all of the messages for that folder) and figure out where every message begins and ends. Thunderbird can't use the so-called index file (folder_name.msf) to do this. Once Thunderbird starts to lose track of where each message begins and ends, compacting can cause the physical removal of other messages because it thinks other messages are part of the deleted message. It's unusual but possible to lose every message in a folder by compacting it if the folder was badly corrupted.

If you suspect a folder is badly corrupted, try to fix it by following the instructions in compacting does not seem to work instead.

Problems when compacting

Compacting seems to take forever

Usually it only takes a couple of minutes to compact your folders unless your profile is over a gigabyte in size or your computer is slow. The worst that normally happens if you exit Thunderbird before it finishes compacting is that it fails to delete some nstmp temporary files that it created and you have to manually delete them.

If compacting takes a long time, the status bar at the bottom of the window may not remove the text about it compacting a folder when it finishes. It may also fail to update that text to mention the last folder in that account if you use "File -> Compact Folders". If you do something else that normally displays some text in the status bar, it should replace the message about compacting.

Compacting does not seem to work

If compacting folders doesn't seem to work even for a single folder (where you right-click on the folder and choose "Compact" from the context menu), it is possible that the folders and/or the .msf mail summary files are corrupted.

Quick and dirty fix

If the corruption is mild you frequently can fix it by deleting the .msf files for the corrupt folders. There are two ways you can do this.

1. Delete the .msf file(s)

  • Exit Thunderbird.
  • Find your profile. If you can not see it read this article .
  • Make a temporary backup of it.
  • Delete all of the files ending in .msf in your profile. Thunderbird will recreate them when it starts.

2. Rebuild the index file within Thunderbird

Right click on each corrupt folder in Thunderbird, select Properties, and then press the Rebuild Index button. Its quicker/simpler if you only have a problem with a few folders, such as the Inbox.

Real fix

If the folder is severely corrupted, deleting the .msf file won't help much, and compacting the folder may just make things worse because Thunderbird can't reliably figure out where a message ends anymore. You'll typically run into this only with the Inbox folder. It's much more vulnerable to corruption because it frequently has both lots of messages and a high percentage of deleted (but not visible) messages. That's why it's recommended that you move messages you want to keep to other folders/child folders rather than keep storing them in the Inbox. You can fix the problem by replacing the corrupt folder with a new known good folder. These instructions assume the Inbox folder is corrupt but you can adapt them for any folder.

  1. Exit Thunderbird and backup your profile folder using something like MozBackup .
  2. Start Thunderbird.
  3. Move all of the messages in the Inbox to other folders/child folders. You can select all of the messages using Control-A. If you have problems moving all of the messages, do it in multiple steps, where you select only a few messages each time.
  4. Verify that all of the messages have been moved.
  5. Exit Thunderbird and delete BOTH the "inbox." and "inbox.msf" files for that account. Leave "Inbox.sbd" alone. You can find where the Inbox folder is stored by looking at Tools -> Account Settings -> Server Settings -> Local Directory, or Tools -> Account Settings -> Local Folders -> Local Directory if you are using a Global Inbox. Its at the bottom of that pane, next to a Browse button.
  6. Start Thunderbird. It will automatically create a new Inbox folder.

While its possible to move the messages back to the (new) Inbox, its recommended you don't do that.

Deleting attachments

Deleting an attachment doesn't immediately decrease the size of the mbox file used to store the messages for that folder. In fact it increases it.

When you delete an attachment it isn't physically deleted. Thunderbird marks the original message as deleted and creates a copy without the attachment. The copy doesn't have the attachment but has headers recording the fact that there used to be an attachment. This extra copy is what temporarily increases the size. You only see the copy because messages marked for deletion are automatically hidden from view.

When you compact the folder the original message is physically deleted, and only the copy without the attachment is left. This reduces the size of the mbox file.

See also

External links