Compacting folders: Difference between revisions

From MozillaZine Knowledge Base
Jump to navigationJump to search
mNo edit summary
 
(35 intermediate revisions by 6 users not shown)
Line 1: Line 1:
{{Tbsuite}}
{{Tbsuite}}


When you delete messages in an email program ("email client") or move them to another folder in the program, they are not yet physically removed - even emptying the Trash does not remove them. Instead, Thunderbird and other email programs simply hide the "deleted" messages and mark them as ready for physical removal. The process of physically removing such no longer visible messages is called "compacting". This means that messages that you think you have deleted are not actually physically removed until you manually or automatically compact the folder they were (are) in.  
When you delete messages in an email client such as Thunderbird they aren't physically deleted. Even emptying the Trash does not get rid of them. Instead they are marked for deletion and hidden from view. They are not physically removed until you "compact" the folder. This is a tradeoff done to improve performance in large folders.


'''If you don't compact your mail folders periodically, they can grow very large and make Thunderbird slow to a crawl. If you wait too long before compacting, you can also lose the entire contents of one or more folders.''' The reason many users have never heard of compacting is that most email programs default to automatically compacting the folder whenever a certain amount of space is wasted, whereas the user has to enable this feature in Thunderbird.
Messages in a Trash folder aren't actually deleted messages, when you delete a message in a POP account it copies the message to the Trash folder and then deletes the original. When you empty the trash it physically removes (expunges) the messages in the Trash folder. You can't undo that.  


Compacting shouldn't be confused with compressing or [http://en.wikipedia.org/wiki/Zip_file zipping] a file. Compressing/zipping only changes ''how'' data is stored; compacting instead changes the data itself.  
"Compacting" a folder has nothing to do with compressing or [http://en.wikipedia.org/wiki/Zip_file zipping] a file. Compressing/zipping only changes ''how'' data is stored; compacting changes the actual data. Compacting a folder frees wasted space but the main reason to do it regularly is because it's a type of preventative maintenance. If you don't compact your folders regularly, performance can slow to a crawl, you can't store new messages anymore, start losing parts of messages or have deleted messages resurrected. If you notice a message has a date of 1969 that is due to it losing the Date: header in the message.  


'''If you suspect a folder is badly corrupted, don't compact it'''. Instead, try to fix it by following the instructions in [[Compacting_folders#Compacting does not seem to work | compacting does not seem to work]].
Corrupt folders tend to occur when you have lots of messages in a folder, many of them are deleted, and you don't compact often enough. This is usually only a problem for the inbox folder as while other folders may get very large you typically don't delete messages in them often. It's recommended that you keep the number of messages in your inbox small, and move any you want to permanently keep to other folders/child folders.


==How to compact folders==
==How to compact folders==


It is best to first turn off any [http://en.wikipedia.org/wiki/Defragmentation defragmentation] program possibly running automatically or manually on your computer because this can cause folder corruption and data loss if it runs simultaneously with Thunderbird's compacting process.
It's recommended that you configure Thunderbird to automatically compact folders rather than try to guess how often is often enough. Automatic is the default starting in version 5. However, a few people have problems due to automatic compacting while something else is happening, if this occurs switch to working offline using File -> Offline -> Work Offline, select an account and then manually compact the folders in that account using File -> Compact Folders instead. You can also compact a specific folder by right clicking on it and selecting "Compact" from the context menu.


===Automatically===
===Automatically===
The easiest way to compact is to let Thunderbird do it automatically: "[[Menu differences in Windows, Linux, and Mac|Tools -> Options]] -> Advanced -> Network & Disk Space -> Disk Space -> Compact folder when it will save over 100 kB -> OK."
The easiest way to compact is to let Thunderbird do it automatically. If you are running version 5 or newer [[Menu differences in Windows, Linux, and Mac|Tools -> Options]]  -> Advanced -> Network & Disk Space -> "Compact folders when it will save over X MB" ''(Thunderbird)'' or [[Menu differences in Windows, Linux, and Mac|Edit -> Preferences]] -> Mail & Newsgroups -> Network & Storage -> "Compact all folders when it will save over X MB in total" ''(SeaMonkey)'' is automatically enabled and defaults to 20MB. If you are running versions prior to Thunderbird 5, check the checkbox for "[[Menu differences in Windows, Linux, and Mac|Tools -> Options]] -> Advanced -> Network & Disk Space -> Disk Space -> Compact folder when it will save over 100 kB" and then press the OK button.  


However, it is best to not to do anything except read messages if you notice that compacting has started. If you tag, mark, or move messages during compacting, this can cause folder corruption and data loss. In addition, if you are in the process of writing an email while compacting starts, you may get an error when you try to save or send it. (This can be especially annoying if you're responding to an email with interleaved replies because copy-pasting to a new message will often move quotation marks into the middle of lines in quoted sections).
See this link for menu paths in various operating systems:  http://kb.mozillazine.org/Menu_differences_in_Windows,_Linux,_and_Mac


If you often delete many messages in very large folders and feel that Thunderbird becomes unresponsive while trying to delete more messages, you may want to try using a larger figure. This will reduce the chance that compacting starts and locks up the folder while you are trying to delete more messages. If you use a figure much larger than 1000 kB, compacting will take much longer and there is a greater risk of folder corruption.
Some people prefer to use a large number to reduce how often they run into a delay due to Thunderbird compacting folders. Others prefer to use a smaller number since it takes far less time to compact when you compact often. Regardless of which approach you choose keep in mind that it's not just a question of wasted space, you want to avoid having too large a percentage of deleted messages in the folder to help reduce the chance of it becoming corrupt. 1MB is usually recommended, and in recent versions it is the minimum possible value.


===Manually===
This limit used to be stored in '''mail.purge_threshhold''', in KB. Thunderbird 5 and later versions ''(and SeaMonkey 2.2 and later)'' ignore that setting and use '''mail.purge_threshhold_mb''' in MB instead. (see [https://bugzilla.mozilla.org/show_bug.cgi?id=437657 bug 437657]).
If you are annoyed by Thunderbird suddenly becoming unresponsive while using it, you may want to turn off automatic compacting and instead compact manually. If you want to combine the advantages of both by making Thunderbird ask for permission before it starts compacting automatically, see [[Compacting folders#Customizing compacting|customizing compacting]] below.


To compact all folders in an account manually, click the account on the left, and then click "File -> Compact Folders". Compacting an account may take from a few seconds to 10 minutes or more, depending on how much mail you have and how recently you last compacted the folders. If you have trouble doing this and the process stalls, try compacting one folder at a time by right-clicking on the folder and choosing "Compact This Folder". If you do not let Thunderbird compact automatically, you should do this regularly, at least once a week. If you do it daily, it will take only a fraction of the time.
'''mail.purge.ask''' defaults to '''false''' in recent versions of Thunderbird and of SeaMonkey, to disable being prompted if it's okay to automatically compact the folder. In old versions you may need to modify that using the Preferences or Options dialog (see above), Thunderbird's [[Modify_Thunderbird_settings|config editor]] or SeaMonkey's [[about:config]].


===Offline===
===IMAP===
Most people never have a problem compacting a folder when Thunderbird is online. However, if you get [[Nstmp folders]], that's due to the compacting being interrupted while Thunderbird is downloading new messages. You can avoid this by going offline before compacting (go to "File -> Offline -> Work Offline", or simply click on the icon in the bottom left corner.) <!-- Hid following because not comprehensible; how could going offline possibly cause problems?: It is rare for this to cause other problems, but if you suspect it, experiment with going offline before compacting (ideally on a new known good folder).--> If you have an IMAP account, you can only compact it while you are online. However, this normally doesn't cause a problem since IMAP accounts only download the headers to the hard disk. (If you don't know what kind of account you have, it's probably a POP3 account, which can be compacted offline.)


===Customizing compacting===
You have several additional options if you are using a IMAP account. If you check "Tools -> Account Settings -> Server Settings -> Clean up ("Expunge") Inbox on Exit it will compact the Inbox folder when you exit Thunderbird. If you check "Empty Trash on Exit" it will also empty and compact the Trash folder when you exit.


If you are an experienced computer user, you can use the [[about:config|Config Editor]] to combine the advantages of automatic and manual compacting by making Thunderbird ask for permission before it starts compacting automatically. In Config Editor, type "mail.purge.ask" (without quotes) and double-click to make its value "true". If there is no mail.purge.ask entry, right-click on any entry in the list of preferences, select ''New'' and ''Boolean'', type "mail.purge.ask" (without quotes), and then choose "true".
If you don't like the performance tradeoffs due to automatically compacting when a certain amount of space is wasted you can configure Thunderbird 3 and later versions to auto-expunge messages instead. This will physically remove deleted messages when you have 20 deleted messages. You can also replace 20 with a different limit. See [[Deleting messages in IMAP accounts]] if you want to do that. If you're using [[Offline_folders|offline folders]] and configured to AutoSync (Thunderbird 3.0 and later does this by default) it will expunge the deleted message in both the remote folder and the offline copy of the folder (an mbox file).


===Other ways to compact folders===
===Add-ons===


The [https://addons.mozilla.org/thunderbird/1279/ Xpunge extension] lets you add a button to empty the trash and compact the folders in multiple accounts (in one step). If you use the beta version you can set a [http://www.theodoretegos.net/mozilla/tb/releases/xpunge/beta/index.html timer] to automate that.
''The web page for some of these add-ons state they add a button to the toolbar. However, it really just adds the button to the set of buttons available for the toolbar. The button is not displayed unless you customize the toolbar to actually add the button. You can customize the toolbar by right clicking on it, selecting customize, drag and drop the button to the toolbar, and then press OK.''


The [https://addons.mozilla.org/en-US/thunderbird/search?q=compact&status=4 Mozilla Add-ons web site] has several other extensions that add buttons to compact folders.
The [https://addons.mozilla.org/en-US/thunderbird/addon/xpunge/?src=ss Xpunge] add-on lets you add a button to empty the trash and compact the folders in multiple accounts (in one step). You can also set a [http://www.theodoretegos.net/mozilla/tb/index.html timer] to automate that.


If you have an [[IMAP]] account there are two ways to automate compacting its folders.  
The [https://addons.mozilla.org/en-US/thunderbird/addon/compactfolder/ CompactFolder] add-on lets you add a button to the toolbar to compact the currently selected folder. The [https://addons.mozilla.org/en-US/thunderbird/search?q=compact&status=4 Mozilla Add-ons web site] has several other extensions that add buttons to compact folders.
 
* If you check Tools -> Account Settings -> Server Settings -> Clean up ("Expunge") Inbox on Exit it will compact the Inbox folder when you exit Thunderbird. If you check "Empty Trash on Exit" it will also empty and compact the Trash folder when you exit.
 
* If Tools -> Account Settings -> Server Settings -> "When I delete a message" is set to "Move it to the Trash folder" and you set mail.imap.expunge_after_delete to true using the [[Modify_Thunderbird_settings | Config editor]] Thunderbird will compact the folder immediately after you delete or move a message. Note: Thunderbird ignores the modified mail.imap.expunge_after_delete setting until after you have exited and restarted Thunderbird.


==Undoing compacting==
==Undoing compacting==


You cannot undo compacting, in other words there is no such thing as "uncompacting". Compacting physically removes messages marked for deletion and hidden from view. Once they're gone, they're gone. Don't confuse compacting a folder with compressing or zipping it.
You cannot undo compacting, there is no such thing as "uncompacting". Compacting physically removes messages marked for deletion and hidden from view. Once they're gone, they're gone. Usually when somebody asks how to "uncompact" a folder they lost messages due to compacting. Sometimes the messages were already gone, but that wasn't noticeable because they were still listed in the folder listing. Other times compacting a folder physically removed messages it shouldn't have because the folder was too corrupt.
 
Usually somebody asks how to "uncompact" a folder when they have lost messages due to compacting. You shouldn't lose messages when compacting unless a folder was badly corrupted, in which case you'd probably have already noticed problems such as message fragments, several messages with a date of 1969 (due to the Date: header in a message being deleted), "resurrected" deleted messages, or long startup times because Thunderbird gets caught in a loop parsing (analysing) the inbox.  


'''If you suspect a folder is badly corrupted, don't compact it'''. It might make things worse because compacting depends upon Thunderbird being able to parse the mbox file (the text file named after the folder that contains all of the messages for that folder) and figure out where every message begins and ends. Thunderbird can't use the so-called index file (folder_name.msf) to do this. Once Thunderbird starts to lose track of where each message begins and ends, compacting can cause the removal of other messages besides those marked ready for physical removal. It's unusual but possible to lose every message in a folder by compacting it if the folder was badly corrupted.  
'''If you suspect a folder is badly corrupted, don't compact it.''' It might make things worse because compacting depends upon Thunderbird being able to parse the mbox file (the text file named after the folder that contains all of the messages for that folder) and figure out where every message begins and ends. Thunderbird can't use the so-called index file (folder_name.msf) to do this. Once Thunderbird starts to lose track of where each message begins and ends, compacting can cause the physical removal of other messages because it thinks other messages are part of the deleted message. It's unusual but possible to lose every message in a folder by compacting it if the folder was badly corrupted.  


If you suspect a folder is badly corrupted, try to fix it by following the instructions in [[Compacting_folders#Compacting does not seem to work | compacting does not seem to work]] instead.
If you suspect a folder is badly corrupted, try to fix it by following the instructions in [[Compacting_folders#Compacting does not seem to work|compacting does not seem to work]] instead.


==Problems when compacting==
==Problems when compacting==
=== Compacting seems to take forever===
=== Compacting seems to take forever===


Usually it only takes a couple of minutes to compact your folders unless your profile is over a gigabyte in size or your computer is slow. The worst that normally happens if you exit Thunderbird before it finishes compacting is that it fails to delete some [[Nstmp_folders | nstmp temporary files]] that it created and you have to manually delete them.
Usually it only takes a couple of minutes to compact your folders unless your profile is over a gigabyte in size, or your computer is slow. Factors contributing to making a computer slow include: being on on a laptop with a slow disk or running on battery, antivirus software watching your Thunderbird file activity (it should never do so), having moved your Thunderbird profile or folders from the default location, and inadequate free disk space which results in highly fragmented disk with resulting poor disk performance.


If compacting takes a long time, the status bar at the bottom of the window may not remove the text about it compacting a folder when it finishes. It may also fail to update that text to mention the last folder in that account if you use "File -> Compact Folders". If you do something else that normally displays some text in the status bar, it should replace the message about compacting.
You can terminate or exit Thunderbird before it finishes compacting and the worst that will happen is that it fails to delete some [[Nstmp_folders|nstmp temporary files]] that it created and you have to manually delete them.
 
The status bar at the bottom of the window may not remove the text about it compacting a folder when it finishes. It may also fail to update that text to mention the last folder in that account if you use "File -> Compact Folders". If you do something else that normally displays some text in the status bar, it should replace the message about compacting.


===Compacting does not seem to work===
===Compacting does not seem to work===


If compacting folders doesn't seem to work even for a single folder (where you right-click on the folder and choose "Compact This Folder"), it is possible that the folders and/or the .msf mail summary files are corrupted.  
If compacting folders doesn't seem to work even for a single folder (where you right-click on the folder and choose "Compact" from the context menu), it is possible that the folders and/or the .msf mail summary files are corrupted.  


====Quick and dirty fix====
====Quick and dirty fix====
Line 71: Line 65:


:*Exit Thunderbird.
:*Exit Thunderbird.
:*Find your [[Profile_folder_-_Thunderbird | profile]]. If you can not see it read [[Show_hidden_files_and_folders | this article]] .
:*Find your [[Profile_folder_-_Thunderbird | profile]]. If you can not see it read [[Show_hidden_files_and_folders|this article]] .
:*Make a temporary [[Profile backup | backup]] of it.   
:*Make a temporary [[Profile backup|backup]] of it.   
:*Delete all of the files ending in .msf in your profile. Thunderbird will recreate them when it starts.  
:*Delete all of the files ending in .msf in your profile. Thunderbird will recreate them when it starts.  


2. Rebuild the index file within Thunderbird
2. Rebuild the index file within Thunderbird


:Right click on each corrupt folder in Thunderbird, select Properties, and then press the Rebuild Index button. Its quicker/simpler if you only have a problem with a few folders, such as the Inbox.
:Right click on each corrupt folder in Thunderbird, select Properties, and then press the "Repair Folder" button. It's quicker/simpler if you only have a problem with a few folders, such as the Inbox.


====Real fix====
====Real fix====
If the corruption is severe deleting the .msf files won't help much, and compacting the folder may just make it worse. You'll typically run into this only with the Inbox folder. Its much more vulnerable to corruption because many users tend to store lots of messages in it and they also frequently delete messages in it. Thats why its recommended that you don't permanently store messages in your Inbox folder, move them to other folders. You can fix the problem by replacing the corrupt folder with a new known good folder that you copied the messages to.
If the folder is severely corrupted, deleting the .msf file won't help much, and compacting the folder may just make things worse because Thunderbird can't reliably figure out where a message ends anymore. You'll typically run into this only with the Inbox folder. It's much more vulnerable to corruption because it frequently has both lots of messages and a high percentage of deleted (but not visible) messages. That's why it's recommended that you move messages you want to keep to other folders/child folders rather than keep storing them in the Inbox. You can fix the problem by replacing the corrupt folder with a new known good folder. These instructions assume the Inbox folder is corrupt but you can adapt them for any folder.


# Exit Thunderbird and make a temporary [[Profile backup | backup]] copy of your [[profile folder]] so that you can revert everything easily if necessary.
# Exit Thunderbird and [[Profile backup|backup]] your [[profile folder]] using something like [[MozBackup]].
# Start Thunderbird, create a new folder and name it "TestFolder" (or similar).
# Start Thunderbird.
# In the folder that is giving you problems (e.g. Inbox), select all the messages (highlight one and then press Ctrl+a) and copy them to TestFolder ("Message -> Copy" -> [account name] -> TestFolder"). In extreme cases (if Thunderbird and/or the computer become extremely sluggish), you may have to choose only a few emails at a time (choose one and then press Shift+Page Down or Page Up once or twice).
# Move all of the messages in the Inbox to other folders/child folders. You can select all of the messages using Control-A. If you have problems moving all of the messages, do it in multiple steps, where you select only a few messages each time.
# Verify that all of the messages have been copied to TestFolder.
# Verify that all of the messages have been moved.
# Right-click on TestFolder and choose "Compact This Folder".
# Exit Thunderbird and delete BOTH the "inbox." and "inbox.msf" files for that account. Leave "Inbox.sbd" alone. You can find where the Inbox folder is stored by looking at [[Menu_differences_in_Windows,_Linux,_and_Mac|Tools -> Account Settings]] -> Server Settings -> Local Directory, or Tools -> Account Settings -> Local Folders -> Local Directory if you are using a [[Global Inbox]]. Its at the bottom of that pane, next to a Browse button.
# Go to your profile folder and see if TestFolder looks like it has been compacted. It should be significantly smaller in size than the folder from which you copied the messages.
# Start Thunderbird. It will automatically create a new Inbox folder.
# If it looks like TestFolder can in fact be compacted successfully, exit Thunderbird, and go to your profile folder. Then:
::* Rename the file that was giving you problems (e.g. rename "Inbox" to "InboxOLD") and delete its .msf file (e.g. "Inbox.msf").
::* Rename the "TestFolder" folder to "Inbox" (or whatever the problematic folder was called) and delete "TestFolder.msf".
::* Restart Thunderbird, and use it as usual. If compacting folders seems to work correctly during a test period (such as one week) then you can delete the folder "InboxOLD".


==What is compacting?==
While it's possible to move the messages back to the (new) Inbox, it's recommended you don't do that.
Email programs do not immediately physically remove messages deleted by users and instead only hide them from view because this improves performance when the users delete messages in large folders. This is because email messages are not separate files; instead, they are consecutive parts of one large file (one file per folder in Thunderbird, one file per entire message store in some other programs). If email programs immediately physically removed messages deleted by the user, they would have to rewrite the entire contents of a folder every time the user deletes a single message.  


When a user deletes 10 messages one at a time in a 100MB folder, Thunderbird can quickly hide them by simply modifying a status header in each message, and Thunderbird doesn't need to rewrite the entire folder until later, when the user compacts it (or lets Thunderbird do it automatically). If the user had to wait while Thunderbird rewrote the entire contents of the folder every single time the user deleted a message, the user would have to wait for a total of 10 times as long as it takes to rewrite 100MB. Compacting is a good performance tradeoff for large folders, but for simplicity you can't choose which folders have to get compacted.
====Compact causes message tabs or windows to close====
Compact can cause messages open in tabs or windows to disappear. If this affects your workflow a reasonable approach is a) disable automatic compact, and instead b) perform it periodically without fail at a time convenient to you, like before shutting Thunderbird down.
a) tools | options | advanced | network and disk
b) for each account, do File | Compact Folders


If a folder has a lot of messages, and if messages are frequently deleted in it, and if you don't compact it frequently, this greatly increases the chance of the folder becoming corrupted. This creates message fragments, and may cause erratic behavior as Thunderbird tries to parse (analyse) the file containing the messages for that folder. Usually this is only a problem for the inbox folder. That is why the [[Keep_it_working_-_Thunderbird | keep it working]] article recommends that you keep the inbox folder as empty as possible (by moving to another folder any messages you want to keep) and configure Thunderbird to automatically compact folders.
==Deleting attachments==
Deleting an attachment doesn't immediately decrease the size of the mbox file used to store the messages for that folder. In fact it increases it.  


When you delete an attachment it isn't physically deleted. Thunderbird marks the original message as deleted and creates a copy without the attachment. The copy doesn't have the attachment but has headers recording the fact that there used to be an attachment. This extra copy is what temporarily increases the size. You only see the copy because messages marked for deletion are automatically hidden from view.


==What happens if you do not compact folders?==
When you compact the folder the original message is physically deleted, and only the copy without the attachment is left. This reduces the size of the mbox file.  
 
If you rarely or never compact folders, your mail files will accumulate more and more of the "hidden" messages that have been marked for permanent deletion but have not yet been removed. This can cause a lot of disk space to be used, and it can have a negative effect on Thunderbird&rsquo;s performance.
* Even if a mail folder seems to be empty or nearly empty, the mail file can become very large [http://forums.mozillazine.org/viewtopic.php?t=85306][http://forums.mozillazine.org/viewtopic.php?t=76239][http://forums.mozillazine.org/viewtopic.php?t=84965]. This wastes disk space, and when you [[Thunderbird : FAQs : Backing Up and Restoring | back up]] your mail files for safekeeping, you will waste time backing up all these "hidden" messages as well.
* When downloading messages, you might occasionally get duplicates of messages you&rsquo;ve already received [http://forums.mozillazine.org/viewtopic.php?t=100245].
* Messages that you have deleted or moved to other folders may unexpectedly reappear in their original folder [http://forums.mozillazine.org/viewtopic.php?t=78881][http://forums.mozillazine.org/viewtopic.php?t=87443].
* Your [[Thunderbird : FAQs : Anti-virus Software | anti-virus software]] might detect infected messages that you long ago deleted, even if you emptied the Trash [http://forums.mozillazine.org/viewtopic.php?p=687327#687327].
* It could cause problems when you try to defragment your hard disk [http://forums.mozillazine.org/viewtopic.php?t=83198].
* Your [[Inbox stays blank|Inbox might stay blank]] for minutes.
* The new message count could become much larger than the actual number of new messages. A quick fix is to delete the .msf mail summary file for that folder&mdash;Thunderbird will create a new one the next time you run it&mdash;but this will not work if the folder is badly corrupted.
* Deleted messages might not get moved to the Trash folder. [http://forums.mozillazine.org/viewtopic.php?f=39&t=1416685&start=0]


== See also ==
== See also ==
* [[Edit large mbox files]]  
* [[Edit large mbox files]]  
* [[Recover messages from a corrupt folder]]
* [[Recover messages from a corrupt folder]]
* [[Nstmp_folders | Why do I have nstmp folders?]]
* [[Nstmp_folders|Why do I have nstmp folders?]]


==External links==
==External links==
* [http://thunderbirdtweaks.blogspot.com.au/2011/07/compacting-what-is-it-and-why-must-i-do.html Compacting :- What is it and Why must I do it. (blog post)]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=286888#c47 Always make compacting folders automatic, with no UI] bug report. It advocates replacing the current way of compacting a folder (copy all of the messages to a new folder file and replace the original folder file) with just moving messages within the folder file in order to avoid blocking other operations.
* [http://www.microsoft.com/windows/IE/community/columns/filecorruption.mspx Compacting and Corruption with Outlook Express]
* [http://www.microsoft.com/windows/IE/community/columns/filecorruption.mspx Compacting and Corruption with Outlook Express]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=466730 Compact folders doesn't compact IMAP offline folders] (fixed in trunk build/3.0)
* [https://bugzilla.mozilla.org/show_bug.cgi?id=466730 Compact folders doesn't compact IMAP offline folders] (fixed in trunk build/3.0)
* [http://journal.mozdev.org/mailtweak.html Mail Tweak] extension   
* [http://journal.mozdev.org/mailtweak.html Mail Tweak] extension   
* [http://www.extensionsmirror.nl/index.php?showtopic=442&hl=purge MoreButtonsPlus] extension.  
* [http://www.extensionsmirror.nl/index.php?showtopic=442&hl=purge MoreButtonsPlus] extension.  
* [http://www.theodoretegos.net/mozilla/tb/index.html Xpunge] extension.  
* [https://www.bugzilla.mozilla.org/show_bug.cgi?id=655469 Provide "Auto-Expunge" options (mail.imap.expunge_option) directly in TB's UI (not just as hidden pref)] bug report.
* [https://nic-nac-project.org/~kaosmos/index-en.html#foldersize ShowFolderSize] shows the folders file size, turning red when it exceeds the limit.
* [http://www.theodoretegos.net/mozilla/tb/index.html Xpunge] extension lets you set a timer for when compacting occurs.  
* [http://www.eyrich-net.org/mozilla/X-Mozilla-Status.html?en X-Mozilla-Status headers] used to store the messages status (such as whether its deleted).
* [http://www.eyrich-net.org/mozilla/X-Mozilla-Status.html?en X-Mozilla-Status headers] used to store the messages status (such as whether its deleted).


[[Category:Mail (Thunderbird)]]  
[[Category:Mail (Thunderbird)]]  
[[Category:Issues (Thunderbird)]]
[[Category:Issues (Thunderbird)]]
[[Category:Configuration (Thunderbird)]]
[[Category:Configuration (Thunderbird)]]

Latest revision as of 21:14, 14 May 2017

This article was written for Thunderbird but also applies to Mozilla Suite / SeaMonkey (though some menu sequences may differ).

When you delete messages in an email client such as Thunderbird they aren't physically deleted. Even emptying the Trash does not get rid of them. Instead they are marked for deletion and hidden from view. They are not physically removed until you "compact" the folder. This is a tradeoff done to improve performance in large folders.

Messages in a Trash folder aren't actually deleted messages, when you delete a message in a POP account it copies the message to the Trash folder and then deletes the original. When you empty the trash it physically removes (expunges) the messages in the Trash folder. You can't undo that.

"Compacting" a folder has nothing to do with compressing or zipping a file. Compressing/zipping only changes how data is stored; compacting changes the actual data. Compacting a folder frees wasted space but the main reason to do it regularly is because it's a type of preventative maintenance. If you don't compact your folders regularly, performance can slow to a crawl, you can't store new messages anymore, start losing parts of messages or have deleted messages resurrected. If you notice a message has a date of 1969 that is due to it losing the Date: header in the message.

Corrupt folders tend to occur when you have lots of messages in a folder, many of them are deleted, and you don't compact often enough. This is usually only a problem for the inbox folder as while other folders may get very large you typically don't delete messages in them often. It's recommended that you keep the number of messages in your inbox small, and move any you want to permanently keep to other folders/child folders.

How to compact folders

It's recommended that you configure Thunderbird to automatically compact folders rather than try to guess how often is often enough. Automatic is the default starting in version 5. However, a few people have problems due to automatic compacting while something else is happening, if this occurs switch to working offline using File -> Offline -> Work Offline, select an account and then manually compact the folders in that account using File -> Compact Folders instead. You can also compact a specific folder by right clicking on it and selecting "Compact" from the context menu.

Automatically

The easiest way to compact is to let Thunderbird do it automatically. If you are running version 5 or newer Tools -> Options -> Advanced -> Network & Disk Space -> "Compact folders when it will save over X MB" (Thunderbird) or Edit -> Preferences -> Mail & Newsgroups -> Network & Storage -> "Compact all folders when it will save over X MB in total" (SeaMonkey) is automatically enabled and defaults to 20MB. If you are running versions prior to Thunderbird 5, check the checkbox for "Tools -> Options -> Advanced -> Network & Disk Space -> Disk Space -> Compact folder when it will save over 100 kB" and then press the OK button.

See this link for menu paths in various operating systems: http://kb.mozillazine.org/Menu_differences_in_Windows,_Linux,_and_Mac

Some people prefer to use a large number to reduce how often they run into a delay due to Thunderbird compacting folders. Others prefer to use a smaller number since it takes far less time to compact when you compact often. Regardless of which approach you choose keep in mind that it's not just a question of wasted space, you want to avoid having too large a percentage of deleted messages in the folder to help reduce the chance of it becoming corrupt. 1MB is usually recommended, and in recent versions it is the minimum possible value.

This limit used to be stored in mail.purge_threshhold, in KB. Thunderbird 5 and later versions (and SeaMonkey 2.2 and later) ignore that setting and use mail.purge_threshhold_mb in MB instead. (see bug 437657).

mail.purge.ask defaults to false in recent versions of Thunderbird and of SeaMonkey, to disable being prompted if it's okay to automatically compact the folder. In old versions you may need to modify that using the Preferences or Options dialog (see above), Thunderbird's config editor or SeaMonkey's about:config.

IMAP

You have several additional options if you are using a IMAP account. If you check "Tools -> Account Settings -> Server Settings -> Clean up ("Expunge") Inbox on Exit it will compact the Inbox folder when you exit Thunderbird. If you check "Empty Trash on Exit" it will also empty and compact the Trash folder when you exit.

If you don't like the performance tradeoffs due to automatically compacting when a certain amount of space is wasted you can configure Thunderbird 3 and later versions to auto-expunge messages instead. This will physically remove deleted messages when you have 20 deleted messages. You can also replace 20 with a different limit. See Deleting messages in IMAP accounts if you want to do that. If you're using offline folders and configured to AutoSync (Thunderbird 3.0 and later does this by default) it will expunge the deleted message in both the remote folder and the offline copy of the folder (an mbox file).

Add-ons

The web page for some of these add-ons state they add a button to the toolbar. However, it really just adds the button to the set of buttons available for the toolbar. The button is not displayed unless you customize the toolbar to actually add the button. You can customize the toolbar by right clicking on it, selecting customize, drag and drop the button to the toolbar, and then press OK.

The Xpunge add-on lets you add a button to empty the trash and compact the folders in multiple accounts (in one step). You can also set a timer to automate that.

The CompactFolder add-on lets you add a button to the toolbar to compact the currently selected folder. The Mozilla Add-ons web site has several other extensions that add buttons to compact folders.

Undoing compacting

You cannot undo compacting, there is no such thing as "uncompacting". Compacting physically removes messages marked for deletion and hidden from view. Once they're gone, they're gone. Usually when somebody asks how to "uncompact" a folder they lost messages due to compacting. Sometimes the messages were already gone, but that wasn't noticeable because they were still listed in the folder listing. Other times compacting a folder physically removed messages it shouldn't have because the folder was too corrupt.

If you suspect a folder is badly corrupted, don't compact it. It might make things worse because compacting depends upon Thunderbird being able to parse the mbox file (the text file named after the folder that contains all of the messages for that folder) and figure out where every message begins and ends. Thunderbird can't use the so-called index file (folder_name.msf) to do this. Once Thunderbird starts to lose track of where each message begins and ends, compacting can cause the physical removal of other messages because it thinks other messages are part of the deleted message. It's unusual but possible to lose every message in a folder by compacting it if the folder was badly corrupted.

If you suspect a folder is badly corrupted, try to fix it by following the instructions in compacting does not seem to work instead.

Problems when compacting

Compacting seems to take forever

Usually it only takes a couple of minutes to compact your folders unless your profile is over a gigabyte in size, or your computer is slow. Factors contributing to making a computer slow include: being on on a laptop with a slow disk or running on battery, antivirus software watching your Thunderbird file activity (it should never do so), having moved your Thunderbird profile or folders from the default location, and inadequate free disk space which results in highly fragmented disk with resulting poor disk performance.

You can terminate or exit Thunderbird before it finishes compacting and the worst that will happen is that it fails to delete some nstmp temporary files that it created and you have to manually delete them.

The status bar at the bottom of the window may not remove the text about it compacting a folder when it finishes. It may also fail to update that text to mention the last folder in that account if you use "File -> Compact Folders". If you do something else that normally displays some text in the status bar, it should replace the message about compacting.

Compacting does not seem to work

If compacting folders doesn't seem to work even for a single folder (where you right-click on the folder and choose "Compact" from the context menu), it is possible that the folders and/or the .msf mail summary files are corrupted.

Quick and dirty fix

If the corruption is mild you frequently can fix it by deleting the .msf files for the corrupt folders. There are two ways you can do this.

1. Delete the .msf file(s)

  • Exit Thunderbird.
  • Find your profile. If you can not see it read this article .
  • Make a temporary backup of it.
  • Delete all of the files ending in .msf in your profile. Thunderbird will recreate them when it starts.

2. Rebuild the index file within Thunderbird

Right click on each corrupt folder in Thunderbird, select Properties, and then press the "Repair Folder" button. It's quicker/simpler if you only have a problem with a few folders, such as the Inbox.

Real fix

If the folder is severely corrupted, deleting the .msf file won't help much, and compacting the folder may just make things worse because Thunderbird can't reliably figure out where a message ends anymore. You'll typically run into this only with the Inbox folder. It's much more vulnerable to corruption because it frequently has both lots of messages and a high percentage of deleted (but not visible) messages. That's why it's recommended that you move messages you want to keep to other folders/child folders rather than keep storing them in the Inbox. You can fix the problem by replacing the corrupt folder with a new known good folder. These instructions assume the Inbox folder is corrupt but you can adapt them for any folder.

  1. Exit Thunderbird and backup your profile folder using something like MozBackup.
  2. Start Thunderbird.
  3. Move all of the messages in the Inbox to other folders/child folders. You can select all of the messages using Control-A. If you have problems moving all of the messages, do it in multiple steps, where you select only a few messages each time.
  4. Verify that all of the messages have been moved.
  5. Exit Thunderbird and delete BOTH the "inbox." and "inbox.msf" files for that account. Leave "Inbox.sbd" alone. You can find where the Inbox folder is stored by looking at Tools -> Account Settings -> Server Settings -> Local Directory, or Tools -> Account Settings -> Local Folders -> Local Directory if you are using a Global Inbox. Its at the bottom of that pane, next to a Browse button.
  6. Start Thunderbird. It will automatically create a new Inbox folder.

While it's possible to move the messages back to the (new) Inbox, it's recommended you don't do that.

Compact causes message tabs or windows to close

Compact can cause messages open in tabs or windows to disappear. If this affects your workflow a reasonable approach is a) disable automatic compact, and instead b) perform it periodically without fail at a time convenient to you, like before shutting Thunderbird down. a) tools | options | advanced | network and disk b) for each account, do File | Compact Folders

Deleting attachments

Deleting an attachment doesn't immediately decrease the size of the mbox file used to store the messages for that folder. In fact it increases it.

When you delete an attachment it isn't physically deleted. Thunderbird marks the original message as deleted and creates a copy without the attachment. The copy doesn't have the attachment but has headers recording the fact that there used to be an attachment. This extra copy is what temporarily increases the size. You only see the copy because messages marked for deletion are automatically hidden from view.

When you compact the folder the original message is physically deleted, and only the copy without the attachment is left. This reduces the size of the mbox file.

See also

External links