Archiving old email – part three

When searching is what you need to do, Google is hard to beat. For this reason, I wanted to mirror my local email archive onto a Gmail account. As I mentioned earlier, I’ve already archived all of my mail locally in Thunderbird. I set up a second Gmail account for my archive, in order to keep my old archive mail separate from the current mail. Since I have mail from several accounts funneling into Thunderbird (including from my main Gmail account), uploading all of this mail back into my main account would be confusing and result in duplicate messages being sent to Gmail. Likewise, I don’t want all of the old email addresses found in my old mail cluttering up the address book of my main Gmail account.

Despite the fact that Google now supports IMAP, it hasn’t gone as smoothly as I’d like. For those of you who don’t know, IMAP is a protocol that lets you upload, download and synchronize mail from your client application (on your PC) to a mail account stored on a server. In theory, I should be able to just set Thunderbird to use my Gmail account using IMAP and upload the mail that way, but in practice, Gmail isn’t very cooperative.

Here’s what I’ve tried:

1. Drag and drop my whole mail folder structure from the local folders to Gmail over IMAP in Thunderbird
2. Upload the mail folders using Outlook instead of Thunderbird
3. Upload each mbox file to Gmail using IMAPSize
4. Move the messages from another IMAP server to Gmail using imapsync

Basically, each attempt encountered problems during the upload. For some reason or another, the transfer would abort, but the problem was always on the server side. I did some reading, and I discovered that Gmail doesn’t officially support uploading via IMAP. Apparently I’m not the only one having problems with this upload.

Now, for what it’s worth, uploading from Thunderbird to Gmail was working, albeit sporadically. If I uploaded small enough chunks infrequently enough to keep Gmail’s servers happy, the upload mostly worked. The good news is that Gmail handles duplicates pretty well. Say you have a folder of 1000 messages, you try to upload it, and it croaks after 400 messages. If you try to upload that folder again, it will NOT duplicate those 400 messages. If you’re patient, just upload one folder at a time using this method.

I wasn’t patient enough for that, so I used another technique, which may or may not work for you. Through my web host, I get to set up IMAP accounts tied to my domain. What I did was pretty simple:

1. Set up a new account IMAP on atterberry.net
2. Upload all my mail to THAT account via IMAP (which wasn’t as picky, since it’s not Gmail)
3. Set up my Gmail account to download all the mail from that atterberry.net via POP
4. Wait!

All that processing just finished a couple of hours ago. Now I have a great searchable archive! I also like the way Gmail assembles the individual emails into conversations.

Now, to keep my archive up-to-date, I need to copy any new mail that I receive in Thunderbird to this archive Gmail account. Since this is usually pretty small, I’ve had pretty good luck uploading to Gmail directly over IMAP.