Compacting folders: Difference between revisions

From MozillaZine Knowledge Base
Jump to navigationJump to search
(Rewrote most of article to tone down the paranoia, shorten it (and make it more readable), and update it for 6.0)
Line 40: Line 40:
You cannot undo compacting, there is no such thing as "uncompacting". Compacting physically removes messages marked for deletion and hidden from view. Once they're gone, they're gone. Usually when somebody asks how to "uncompact" a folder they lost messages due to compacting. Sometimes the messages were already gone, but that wasn't noticeable because they were still listed in the folder listing. Other times compacting a folder physically removed messages it shouldn't have because the  folder was too corrupt.
You cannot undo compacting, there is no such thing as "uncompacting". Compacting physically removes messages marked for deletion and hidden from view. Once they're gone, they're gone. Usually when somebody asks how to "uncompact" a folder they lost messages due to compacting. Sometimes the messages were already gone, but that wasn't noticeable because they were still listed in the folder listing. Other times compacting a folder physically removed messages it shouldn't have because the  folder was too corrupt.


'''If you suspect a folder is badly corrupted, don't compact it'''. It might make things worse because compacting depends upon Thunderbird being able to parse the mbox file (the text file named after the folder that contains all of the messages for that folder) and figure out where every message begins and ends. Thunderbird can't use the so-called index file (folder_name.msf) to do this. Once Thunderbird starts to lose track of where each message begins and ends, compacting can cause the physical removal of other deleted messages because it thinks other messages are part of the deleted message. It's unusual but possible to lose every message in a folder by compacting it if the folder was badly corrupted.  
'''If you suspect a folder is badly corrupted, don't compact it'''. It might make things worse because compacting depends upon Thunderbird being able to parse the mbox file (the text file named after the folder that contains all of the messages for that folder) and figure out where every message begins and ends. Thunderbird can't use the so-called index file (folder_name.msf) to do this. Once Thunderbird starts to lose track of where each message begins and ends, compacting can cause the physical removal of other messages because it thinks other messages are part of the deleted message. It's unusual but possible to lose every message in a folder by compacting it if the folder was badly corrupted.  


If you suspect a folder is badly corrupted, try to fix it by following the instructions in [[Compacting_folders#Compacting does not seem to work | compacting does not seem to work]] instead.
If you suspect a folder is badly corrupted, try to fix it by following the instructions in [[Compacting_folders#Compacting does not seem to work | compacting does not seem to work]] instead.

Revision as of 06:54, 11 September 2011

This article was written for Thunderbird but also applies to Mozilla Suite / SeaMonkey (though some menu sequences may differ).

When you delete messages in a email client such as Thunderbird they aren't physically deleted. Even emptying the Trash does not get rid of them. Instead they are marked for deletion and hidden from view. They are not physically removed until you "compact" the folder. This is a tradeoff done to improve performance in large folders.

"Compacting" a folder has nothing to do with compressing or zipping a file. Compressing/zipping only changes how data is stored; compacting changes the actual data. Compacting a folder frees wasted space but the main reason to do it regularly is because its a type of preventative maintenance. If you don't compact your folders regularly performance can slow to a crawl, you can't store new messages anymore, start losing parts of messages or have deleted messages resurrected. If you notice a message has a date of 1969 that is due to it losing the Date: header in the message.

Corrupt folders tend to occur when you have lots of messages in a folder, many of them are deleted, and you don't compact often enough. This is usually only a problem for the inbox folder as while other folders may get very large you typically don't delete messages in them often. Its recommended that you keep the number of messages in your inbox small, and move any you want to permanently keep to other folders/child folders.

How to compact folders

Its recommended that you configure Thunderbird to automatically compact folders rather than try to guess how often is often enough. A few people have problems due to automatic compacting while something else is happening, if this occurs switch to working offline using File -> Offline -> Work Offline, select an account and then manually compact the folders in that account using File -> Compact Folders instead. You can also compact a specific folder by right clicking on it and selecting "Compact" from the context menu.

Automatically

The easiest way to compact is to let Thunderbird do it automatically. If you are running Thunderbird 3.1.14 or an earlier version check the checkbox for "Tools -> Options -> Advanced -> Network & Disk Space -> Disk Space -> Compact folder when it will save over 100 kB" and then press the OK button. If you are running a more recent version Tools -> Options -> Advanced -> Network & Disk Space -> "Compact folders when it will save over X MB" is used instead, and its automatically enabled. "X" defaults to 20MB in Thunderbird 5.0.

Some people prefer to use a large number to reduce how often they run into a delay due to Thunderbird compacting folders. Others prefer to use a smaller number since it takes far less time to compact when you compact often. Regardless of which approach you choose keep in mind that its not just a question of wasted space, you want to avoid having too large a percentage of deleted messages in the folder to help reduce the chance of it becoming corrupt. 1MB is usually recommended.

This limit used to be stored in mail.purge_threshhold, in KB. Thunderbird 5 and later versions ignore that setting and use mail.purge_threshhold_mb in MB instead. [1]

Set mail.purge.ask false using the config editor to disable being prompted if its okay to compact the folder.

IMAP

You have several additional options if you are using a IMAP account. If you check "Tools -> Account Settings -> Server Settings -> Clean up ("Expunge") Inbox on Exit it will compact the Inbox folder when you exit Thunderbird. If you check "Empty Trash on Exit" it will also empty and compact the Trash folder when you exit.

If you don't like the performance tradeoffs due to automatically compacting when a certain amount of space is wasted you can configure Thunderbird 3 and later versions to auto-expunge messages instead. This will physically remove deleted messages when you have 20 deleted messages. You can also replace 20 with a different limit. See Deleting messages in IMAP accounts if you want to do that. If you're using offline folders and configured to AutoSync (Thunderbird 3.0 and later does this by default) it will expunge the deleted message in both the remote folder and the offline copy of the folder (an mbox file).

Add-ons

The Xpunge extension lets you add a button to empty the trash and compact the folders in multiple accounts (in one step). If you use the beta version you can set a timer to automate that.

The CompactFolder add-on lets you add a button to the toolbar to compact the currently selected folder. You can customize the toolbar by right clicking on it, selecting customize, drag and drop the button to the toolbar, and then press OK.

The MailTweak add-on has a "Compact from folder properties" tweak. It adds a Compact button and shows the unused space in the folder file as a percentage of its size on disk in the folders properties. It also disables Compact in the context menu if the folder is already compacted.

The Mozilla Add-ons web site has several other extensions that add buttons to compact folders.

Undoing compacting

You cannot undo compacting, there is no such thing as "uncompacting". Compacting physically removes messages marked for deletion and hidden from view. Once they're gone, they're gone. Usually when somebody asks how to "uncompact" a folder they lost messages due to compacting. Sometimes the messages were already gone, but that wasn't noticeable because they were still listed in the folder listing. Other times compacting a folder physically removed messages it shouldn't have because the folder was too corrupt.

If you suspect a folder is badly corrupted, don't compact it. It might make things worse because compacting depends upon Thunderbird being able to parse the mbox file (the text file named after the folder that contains all of the messages for that folder) and figure out where every message begins and ends. Thunderbird can't use the so-called index file (folder_name.msf) to do this. Once Thunderbird starts to lose track of where each message begins and ends, compacting can cause the physical removal of other messages because it thinks other messages are part of the deleted message. It's unusual but possible to lose every message in a folder by compacting it if the folder was badly corrupted.

If you suspect a folder is badly corrupted, try to fix it by following the instructions in compacting does not seem to work instead.

Problems when compacting

Compacting seems to take forever

Usually it only takes a couple of minutes to compact your folders unless your profile is over a gigabyte in size or your computer is slow. The worst that normally happens if you exit Thunderbird before it finishes compacting is that it fails to delete some nstmp temporary files that it created and you have to manually delete them.

If compacting takes a long time, the status bar at the bottom of the window may not remove the text about it compacting a folder when it finishes. It may also fail to update that text to mention the last folder in that account if you use "File -> Compact Folders". If you do something else that normally displays some text in the status bar, it should replace the message about compacting.

Compacting does not seem to work

If compacting folders doesn't seem to work even for a single folder (where you right-click on the folder and choose "Compact This Folder"), it is possible that the folders and/or the .msf mail summary files are corrupted.

Quick and dirty fix

If the corruption is mild you frequently can fix it by deleting the .msf files for the corrupt folders. There are two ways you can do this.

1. Delete the .msf file(s)

  • Exit Thunderbird.
  • Find your profile. If you can not see it read this article .
  • Make a temporary backup of it.
  • Delete all of the files ending in .msf in your profile. Thunderbird will recreate them when it starts.

2. Rebuild the index file within Thunderbird

Right click on each corrupt folder in Thunderbird, select Properties, and then press the Rebuild Index button. Its quicker/simpler if you only have a problem with a few folders, such as the Inbox.

3. Compact all folders

Click the account on the left, and then click "File -> Compact Folders". This has to be repeated for each account. If you have trouble doing this and the process stalls, try compacting one folder at a time by right-clicking on the folder and choosing "Compact" ("Compact This Folder" in older versions).

Real fix

If the corruption is severe, deleting the .msf files won't help much, and compacting the folder may just make it worse. You'll typically run into this only with the Inbox folder. It's much more vulnerable to corruption because many users tend to store lots of messages in it, and they also frequently delete messages in it. That's why it's recommended that you don't permanently store messages in your Inbox folder and instead move them to other folders. You can fix the problem by replacing the corrupt folder with a new known good folder into which you copy the messages from the corrupt folder.

  1. Exit Thunderbird and make a temporary backup copy of your profile folder so that you can revert everything easily if necessary.
  2. Start Thunderbird, create a new folder and name it "TestFolder" (or similar).
  3. In the folder that is giving you problems (e.g. Inbox), select all the messages (highlight one and then press Ctrl+a) and copy them to TestFolder ("Message -> Copy" -> [account name] -> TestFolder"). In extreme cases (if Thunderbird and/or the computer become extremely sluggish), you may have to choose only a few emails at a time (choose one and then press Shift+Page Down or Page Up once or twice).
  4. Verify that all of the messages have been copied to TestFolder.
  5. Right-click on TestFolder and choose "Compact" ("Compact This Folder" in older versions).
  6. Go to your profile folder and see if TestFolder looks like it has been compacted. It should be significantly smaller in size than the folder from which you copied the messages.
  7. If it looks like TestFolder can in fact be compacted successfully, exit Thunderbird, and go to your profile folder. Then:
  • Rename the file that was giving you problems (e.g. rename "Inbox" to "InboxOLD") and delete its .msf file (e.g. "Inbox.msf").
  • Rename the "TestFolder" folder to "Inbox" (or whatever the problematic folder was called) and delete "TestFolder.msf".
  • Restart Thunderbird, and use it as usual. If compacting folders seems to work correctly during a test period (such as one week) then you can delete the folder "InboxOLD".

Deleting attachments

Deleting an attachment doesn't immediately decrease the size of the mbox file used to store the messages for that folder. In fact it increases it.

When you delete an attachment it isn't physically deleted. Thunderbird marks the original message as deleted and creates a copy without the attachment. The copy doesn't have the attachment but has headers recording the fact that there used to be an attachment. This extra copy is what temporarily increases the size. You only see the copy because messages marked for deletion are automatically hidden from view.

When you compact the folder the original message is physically deleted, and only the copy without the attachment is left. This reduces the size of the mbox file.

See also

External links