MozillaZine

Talk:Compacting folders

From MozillaZine Knowledge Base

Contents

Compacting offline

The recommendation to only do this is offline is good, but is only applicable to POP3 clients. If you are using IMAP, then your mail is on the server, and you can't compact it while offline. Thunderbird greys out the menu to avoid you trying, but if you're not thinking (like me!) you waste a few minutes trying to work out why the menu is greyed out. [Mozillazine newbie - apologies if I've posted this on the wrong end of the thread!]


Should one recommend doing this only offline? At least in OE, compacting while messages are being downloaded can destroy the whole Inbox, which is why MS has finally got rid of the possibility of having this done automatically ("background compacting").

And if there is no danger of TB corrupting the Inbox by simultaneously downloading and compacting, there is probably a way that at least advanced users would like to automate this via user.js. So far, i've only seen hacks http://forums.mozillazine.org/viewtopic.php?p=994438#994438 to imitate the newer TB versions' function that asks whether this should be done instead of just doing it.

I modified the text to eliminate the recommendation to compact POP accounts offline. Most people never have a problem compacting POP accounts online. I changed the text to recommend it only if you run into problems. Thats a reasonable trade off given how few people have problems, and its almost always just nstmp folders. Tanstaafl 00:11, 9 September 2007 (UTC)

"Drastic" solution if all else fails

22:13, 11 Jan 2005 (Tue):

With apologies to whomever posted it, I pulled this solution:

In rare cases, you may need to look at [this solution.]

The link is to is a forum post which includes:

I noticed that there was a file (not the folder of the same name) in that [profile\mail\account] folder called, simply "Inbox" with no file extension.

A file which the poster then deleted(!). There's a warning later about losing data, but someone who doesn't read carefully could easily lose their whole inbox.

Two concerns:

1) What does this have to do with compressing folders? Deleting a mail folder is not a solution for compressing it. Perhaps this belongs in a different KB article?

2) It's too risky to direct users to that post: At least let's omit the forum post and write instructions to rename the inbox, not delete it, with many caveats to back it up.


To 1) This seems to be the only solution to (hopefully rare!?) cases in which no compacting methods work. As the poster explained, his Inbox was over 2 GB although there were almost no (undeleted) messages in it.

2) You're right. I'd thought the poster's warning* was clear enough, but it's not strong enough. It seems he didn't really know what he was doing despite stumbling across a workaround for a very big problem that has not been getting enough attention.

(*Before you try this, let me make this warning: I do not keep any mail in the inbox so there was little risk of me losing mail by attempting this (that's also how I knew that the 2 gig file size was way off). If you keep mail in your inbox, you may want to move it before trying this.)

What do you think about this?: http://forums.mozillazine.org/viewtopic.php?p=1115988#1115988 Could this be a clue to the reason compacting sometimes doesn't work, and what does it tell us about the "Cannot write email to the mailbox" problem. And is this bug related to the compacting bug?


1) This seems to be the only solution to (hopefully rare!?) cases in which no compacting methods work

Yes, but a solution to what? Unless I misunderstand, the question this answers is how do I get my incoming mail? and not how do I compact folders?. The knowledge base page we're discussing is a about compacting folders.

It's worth adding to the KB, I agree, but on the appropriate page, probably one of these or a new one page linked there.


As i wrote, i believe the two problems are perhaps related. (What do you think about my last paragraph?)

In any case, the emergency solution of deleting the inbox is better than nothing in cases where compacting does not work using any other method and users can no longer see any or some of their inbox messages (often intermittently). It's not as bad as it sounds if the instructions include a clear warning and especially if users are able to first copy all (especially when only intermittently invisible) or at least the important messages out of the Inbox.

And if the two different (but perhaps related) problems of not being able to compact and of not being able to download are not extremely rare, TB should have a prominent message in the UI and in the help file advising users not to use their inbox as a permanent storage and advising them to always move email out of the inbox manually or with filters.

For many if not most people, not being able to get new mail (in this case due to an uncompacted folder) is a bigger problem than losing old mail, especially if they have a recent backup. I would guess that 30% of normal users are capable of making a backup of their profile and of deleting a file (using either Search or %appdata%, not the complicated Folder Options / View method in the chaotic and unnecessarily complicated KB profile article!), but 90% get completely confused by the concepts of creating a new profile or migrating stuff.

Most users will drop TB immediately if they can't get quick help when they can't see (or download) new mail. If they are advised to first make a backup of their profile folder, the salvage work can still be performed later. BTW, the Profile Backup KB article is a joke too because it doesn't explain what it is supposed to for backing up email!




This page is about compacting folders and nothing else. Perhaps you could create pages addressing the other problem(s) (if they doesn't already exist) and include a links between this page and the new one(s)?

For many if not most people, not being able to get new mail (in this case due to an uncompacted folder) is a bigger problem than losing old mail

While I can see the reasoning, I'm not sure I agree. Losing data (dataloss) is usually considered far worse than a loss of function, or even loss of the whole application.


the emergency solution of deleting the inbox is better than nothing in cases where compacting does not work using any other method and users can no longer see any or some of their inbox messages (often intermittently). It's not as bad as it sounds if the instructions include a clear warning and especially if users are able to first copy all (especially when only intermittently invisible) or at least the important messages out of the Inbox.

I agree, with a few changes:

 1)  It belongs on a different page (linked to from the Compacting Folders page)
 2)  *** Don't delete the inbox, rename it ***
 3)  We should instruct users to move *all* messages, not just important ones, to another folder.


I'm afraid I don't have a solution to the other problem. I'm not 100% sure what it is: I see mentions of problems receiving e-mail, users not seeing e-mail in their inbox, and 'cannot write e-mail to mailbox' errors. Maybe if you post it to a forum, you'll find someone who knows.

Thanks for helping out.

"Drastic" solution dumped

While cleaning up the article earlier today, I looked at the paragraph about renaming Inbox to Inbox2 and could not understand why it would solve a problem with compacting folders. Without explanation, it's not at all clear how merely renaming a file would fix anything. I thus completely rewrote that section. --Wintogreen 09:05, 9 May 2005 (PDT)

Compacting the while folder tree

What about information on bulk compacting of the Folder Tree? Is there way to write a 'Compact All Folders' command? I already store all important inbound mail into folders but then when you clean those folders you get slack space and then have to maunally compact eact folder individually.

How is this acceptable behaviour?

It is pretty non-obvious that compacting folders has to be done. I only found out after finding out my profile directory was nearly a gig in size, thanks mostly to spam that I had thought was long ago deleted. At the very least, it should be assumed that when a user empties the trash can, he really, really wants to delete things. That would be a perfect time to also run a compact. Of course, it would be better if it just compacted automatically or the Delete function did what it says on the tin.

If It Won't Compact, Then Try This Act.

Apologies to Johnnie Cochran, RIP.

OK, my problem was that on fireup TBird was taking forever to catalog itself and then get my new mail off the POP server, all because the current Inbox file had grown to huge proportions, despite my moving tons of old emails into archive folders. All efforts to compact Inbox just failed.

The solution for me was a piece of cake. (1) I closed TBird, located the bloated Inbox file under Windows (98SE) Profiles, and renamed it Inbox.OLD. (2) I blocked Internet access (using ZoneAlarm), just to avoid possible complications. (3) I fired up TBird again, and it (a) created a new Inbox for me, and (b) recognized Inbox.OLD as another mail folder. (3) I selected all emails in Inbox.OLD and moved them to the new Inbox. (4) I deleted Inbox.OLD. (5) I got online again, and got my new mail fast.

Voila! File bloat and email delay all gone.

But like the man says...back up first.

Cheers!

A.

I don't understand why there can't be a protection in the software to automatically prevent it from downloading new emails while compacting the folders.

I like to be reminded when it's time to compact folders but that almost always happens when I start up the software. And, of course, that's exactly the moment when the new emails come in...

Or maybe there should be an option where you can have the folders compacted every time you stop the software?

not enuf space on drive

should the disk space requirement be mentioned? the fact that compacting a folder/mailbox requires roughly as much free space on the drive as the size of the folder being compacted might not be obvious. ihave had disk-is-full msgs from the os when compacting upon occasion. -lee-

automatic compacting

2.0 can compact automatically, but i need answers to the questions below before correcting this:

The difference is that most other email clients by default automatically compact the folder when a certain amount of space is wasted; Thunderbird does not do this.

1) Does TB go offline automatically when one tells it to compact automatically? Hopefully, the developers have not made the opposite kind of dangerous behavior the default...

2) How does one get TB to stop compacting automatically if one has put a check mark in "Do this automatically from now on"? If one puts a check mark in there, one never gets that dialog box again, and there is no relevant setting in "network & disk space". --American Finn 18:17, 26 May 2007 (UTC)

I'm not aware of a recent change to make it go offline automatically. I know it definitely didn't in older versions. According to this google group thread mail.purge.ask is the setting you're looking for. However, with 2.0.0.6 I can uncheck the setting in "network & disk space" if I change my mind. Tanstaafl 00:11, 9 September 2007 (UTC)

3) In version 3.3 the default is changing to 20MB, and minimum 1MB via bug 437657. kb article should probably also specify the same minimum. --Wsm 22:03, 28 February 2011 (UTC)

Good point. However I think its a little too early to make any changes since 3.3 has not been released, and anybody using alphas/nightlies better be able to handle such a minor discrepancy in the article. When a change is made we should also mention the prior value since the articles are not meant to be limited to supporting just the latest major version. For example, right now we continue to support 2.0.0.24 but normally remove all mention of earlier releases as they haven't just reached end of life, they're obsolete. Tanstaafl 09:24, 1 March 2011 (UTC)
The new default should be 20MB, effective for TB/Miramar 3.3 alpha 3, and also with the upcoming SeaMonkey 2.1 version. Also, compacting is now switched on by default as a follow-up patch in 437657. --Rsx11m 14:23, 23 May 2011 (UTC)
Also, values are now in MB rather than KB, and existing settings will be migrated. --Rsx11m 14:27, 23 May 2011 (UTC)
Given that 5.0b1 is about to be released soon, I've added a "switched on by default in 5.0" heads-up to the article. Also, "figure much larger than 1000 kB" sounds outdated and I have therefore replaced it with a more general statement. The actual threshold when it becomes annoying likely depends on the hardware used, thus I was reluctant to simply change it to a higher number. --Rsx11m 15:38, 30 May 2011 (UTC)

Can someone help me understand "with a very large threshold, compacting will take much longer and there is a greater risk of folder corruption."? Compacting folder A, at point in time X might take longer at higher value threshold. But, does it not make sense to look at this issue from a long(er) aggregate time period Y than just a point in time X? Bearing in mind ...

  • Value for threshold is across ALL folders, not just one folder like inbox. So unless Inbox is the only folder a user is deleting from, then the actual amount of compact time is spread across multiple folders.
  • If probability of corruption is proportional to the total amount of time that folders are being compacted, and the amount of time is proportional to the size of the folders or the amount data to be compacted out, then the probability of corruption with threshold set at N done M times over period Y should not be greater than compacting with threshold is set at MxN (again, in time period Y). (indeed logically it seems to me that higher value should result in less compaction time in the aggregate, and thus lower the overall probability of corruption)

However, user wait time a different matter ... If a user wants to do something in the UI with a folder, and the compaction triggers because the user is operating in that same folder, then a higher value does potentially cause longer UI wait, which is bad because users have keen memory for worst case experience. (which is why some of the bugs in http://bit.ly/mGT0cS would be great to see fixed, like Bug 398523 - Compact folder ... Should schedule with some delay --Wsm 17:13, 30 May 2011 (UTC)

bienvenu's comment echoes my thoughts. However, is it really true in practice? --Wsm 20:58, 30 May 2011 (UTC)
My experience is that moderators that spend a lot of time in the Thunderbird forums usually recommend a low threshold because we've found (based on user feedback) that more frequent but much shorter delays is usually less annoying. Please keep in mind that we also recommend that the inbox folder be kept fairly empty, and that users move any messages they want to keep to other folders/child folders. That also effects how long the delay is. However, the main problem with his comment is he completely ignored what I was trying to say about thresholds needing to be customized based on what type of mail a person gets. Most of my messages are about 5 KB (the largest message in my main account is 88KB), I've found that a 500KB threshold causes corruption in remote inbox folders. I don't claim I'm typical, I'm just using that as an example of why one size doesn't fit all. I strongly disagree with the idea of us changing our recommendations just because the developers change the default setting. If you want that you can get it at the MoMo web site. Tanstaafl 02:40, 31 May 2011 (UTC)
Thanks. Is there a bug# for your issue "I've found that a 500KB threshold causes corruption in remote inbox folders"? --Wsm 03:24, 31 May 2011 (UTC)
Just a reminder that the new minimum for the auto-compact threshold is 1MB, thus if "something" happens at 500KB already, it would affect all auto-compact actions. You may want to re-test this after the various backend changes made to see if it still applies. --Rsx11m 12:47, 31 May 2011 (UTC)
Good point. In case it's not obvious, I'll help anyone with a reproducible testcase, with expectations of getting such issues resolved, and build on the fixes of the past couple years http://bit.ly/m1qRDe --Wsm 18:24, 31 May 2011 (UTC)
Nope. I didn't see the point since I'm not aware of any spec. that says how many deleted messages etc. you should be able to have in a folder before corruption occurs, and one of the main reasons for Bienvenu's work on adding maildir support is to provide a long term solution to corrupted folders. Tanstaafl 05:07, 31 May 2011 (UTC)
There is no spec - BECAUSE IT SHOULDN'T HAPPEN. But, I wasn't asking if you filed a bug ... I asked if there is a bug# that matches. Is the corruption in the mbox (not the index)? And can you quantify ... is it 50% of the time, 90%? No one can force you, but if you have a clear testcase, it would benefit *other* users if you would pursue it. Besides, I wouldn't be hanging my hat on anything that is taking several years to deliver, has an undefined migration process for users, and still has no clear sign of shipping in a release this year. --Wsm 11:42, 31 May 2011 (UTC)