Korey Atterberry’s Idle Chatter

I’ve got a new building project, and I needed some graph paper to do my design work. You can always go buy a pad of special graph paper, but I found that you can download PDFs of graph paper and print them at home.

There are a lot of choices, but one I like is from Incompetech.com. I use the multi-weight PDF generator to create exactly the grid I need.

Also, there are many choices for free music staff paper as well.

When searching is what you need to do, Google is hard to beat. For this reason, I wanted to mirror my local email archive onto a Gmail account. As I mentioned earlier, I’ve already archived all of my mail locally in Thunderbird. I set up a second Gmail account for my archive, in order to keep my old archive mail separate from the current mail. Since I have mail from several accounts funneling into Thunderbird (including from my main Gmail account), uploading all of this mail back into my main account would be confusing and result in duplicate messages being sent to Gmail. Likewise, I don’t want all of the old email addresses found in my old mail cluttering up the address book of my main Gmail account.

Despite the fact that Google now supports IMAP, it hasn’t gone as smoothly as I’d like. For those of you who don’t know, IMAP is a protocol that lets you upload, download and synchronize mail from your client application (on your PC) to a mail account stored on a server. In theory, I should be able to just set Thunderbird to use my Gmail account using IMAP and upload the mail that way, but in practice, Gmail isn’t very cooperative.

Here’s what I’ve tried:

1. Drag and drop my whole mail folder structure from the local folders to Gmail over IMAP in Thunderbird
2. Upload the mail folders using Outlook instead of Thunderbird
3. Upload each mbox file to Gmail using IMAPSize
4. Move the messages from another IMAP server to Gmail using imapsync

Basically, each attempt encountered problems during the upload. For some reason or another, the transfer would abort, but the problem was always on the server side. I did some reading, and I discovered that Gmail doesn’t officially support uploading via IMAP. Apparently I’m not the only one having problems with this upload.

Now, for what it’s worth, uploading from Thunderbird to Gmail was working, albeit sporadically. If I uploaded small enough chunks infrequently enough to keep Gmail’s servers happy, the upload mostly worked. The good news is that Gmail handles duplicates pretty well. Say you have a folder of 1000 messages, you try to upload it, and it croaks after 400 messages. If you try to upload that folder again, it will NOT duplicate those 400 messages. If you’re patient, just upload one folder at a time using this method.

I wasn’t patient enough for that, so I used another technique, which may or may not work for you. Through my web host, I get to set up IMAP accounts tied to my domain. What I did was pretty simple:

1. Set up a new account IMAP on atterberry.net
2. Upload all my mail to THAT account via IMAP (which wasn’t as picky, since it’s not Gmail)
3. Set up my Gmail account to download all the mail from that atterberry.net via POP
4. Wait!

All that processing just finished a couple of hours ago. Now I have a great searchable archive! I also like the way Gmail assembles the individual emails into conversations.

Now, to keep my archive up-to-date, I need to copy any new mail that I receive in Thunderbird to this archive Gmail account. Since this is usually pretty small, I’ve had pretty good luck uploading to Gmail directly over IMAP.

In the previous post, I described my method for archiving my old email in Thunderbird. As promised, here are some notes about cleaning up your inbox in Thunderbird.

Removing duplicates

One great tool I like to use is the “Remove Duplicate Messages (Alternate)” Thunderbird extension. To use the extension, you must first set the options, then you run it on a folder.

To change the options for the duplicates extension, go to Tools > Addons, find the extension, and hit the “Options” button. You’ll see the dialog shown below:

Setting these checkboxes changes the criteria for comparing two messages. The more boxes you check, the more sure you can be that any duplicates you find will actually be duplicates. I like to start by setting all the checkboxes. This takes some time, but it’s very safe.

To run the extension, right click on a folder and select “Remove Duplicates.” It will crank away for a while then show you a dialog on which you can review and delete the duplicates found. I right click on my “Mail Archive” folder, since that will compare all messages in the subfolders within the archive.

After that pass, depending on how you’ve handled you email in the past, you may need to run the duplicate remover with different options. If you do this, you’ll likely eliminate quite a few more duplicates, but you should always preview the emails in this step before deleting anything. Try turning off the following criteria, one at a time: number of lines, body, message ID and folder. I found some duplicates with minor differences in those areas due to the different ways the messages had been handled in the past. Turning off the folder criteria will find duplicates in different folders (which is nice), but be careful to keep the copy that you’ve filed properly.

Whitelisting

Another technique I found VERY effective in clearing out my email backlog was to set up a “whitelist” in Thunderbird. Basically, I add every valid email address for a real person I’ve communicated with into an address book and set up filters to move messages around based on whether the sender is in that list.

Now, you probably wouldn’t want to add all of these email addresses to your regular address book, but Thunderbird allows you to have multiple address books. I created a new one called “Whitelist” and didn’t put a lot of effort into keeping that one neat or fully filled out. The whitelist address book will contain any email address, current or old. You probably only want active email addresses in your normal address book.

To build my whitelist, I went through my folder of unsorted mail (sorted by sender), and previewed messages from anyone I knew. For each new person, I right-clicked on the email address and chose “Add to Addressbook”. I didn’t bother filling in the person’s name, since it doesn’t really matter for this technique.

The next step is to set up a filter. Create a filter that you can run on your unsorted mail folder like the one shown below.

My filter moves all whitelist email to a folder called “Whitelisted”. It runs both when new mail arrives and when I manually run it on the unsorted inbox folder. With this setup, I can refine my whitelist. If I add some more addresses to the whitelist, I can rerun the filter until I’m left with only junk in my inbox.

After all this processing, I’ve separated my good email (from real people) from the mass mail, etc. At that point, I can move the good email to the archive and take care of the other mail however I see fit.

Some may wonder why, but I’ve been on a long quest to properly archive my old email. I have around 25,000 email messages saved since 1995, and I’d like them to be organized, searchable and future-proof. Of course, some emails I like to keep for purely sentimental reasons, but I also have found my archive useful when working with home videos and photos albums. Have a video of a camping trip but don’t remember when it was? Find the planning emails!

Over the years I’ve had a lot of accounts on a lot of systems. First, in college, I had your typical Unix mail. Later, I used a POP mail account with Eudora. Most recently, I ended up using Outlook to download the mail from my webmail accounts. I wasn’t entirely happy with this setup, since only Outlook can read the PST files that the mail is stored in, and if a PST file gets too big it can self-destruct.

Local mail archive

The first thing I wanted to do was to store my mail in a standard format (i.e. future-proof) on my PC. I chose Thunderbird as my desktop client, since it runs on any operating system, is fairly stable, and uses the standard “mbox” mail file format. Likewise, importing mail from Outlook is easy (there’s an import wizard), and importing from Unix mail and Eudora files is as easy as copying the mailbox files into the mail folder (since both use the mbox format).

Importing your backlog

Here are the steps I took to archive my backlog of old mail in Thunderbird:

1. Install Thunderbird.
2. Import all your old email into Thunderbird. If you’re using Eudora, Outlook or Outlook Express, you can just use Thunderbird’s import feature.
3. Make sure you don’t have any duplicates. Use the Remove Duplicate Messages Thunderbird extension. See part two for some hints.
4. Clean up the spam, junk notifications, etc. from your imported mail. See part two for more information.
5. Move all of the old mail under a “Mail Archive” folder. Create subfolders to organize if you want, but keep it all under the archive folder.

Archiving new mail

To continue archiving the new mail you receive, set up Thunderbird to download your incoming mail. Here’s what I did:

1. Set up Thunderbird to download mail from your POP3 accounts. Here’s how to set up Thunderbird with Gmail, for instance.
2. Make sure you don’t directly put this mail into the “Mail Archive” folder listed above.
3. Set up filters to delete or move any email you’re not going to keep.

Every once in a while, you can process new mail into the archive:

1. Hit the “Get Mail” button in Thunderbird to download new messages into the inbox.
2. Clean up the inbox using all the techniques you used for cleaning up your initial backlog above.
3. Move the good mail to the “Mail Archive” folder once it’s clean.

Those are my basic processes. In a post coming soon I’ll discuss some techniques for cleaning up your mail folders to get rid of spam, notification emails, etc.

Yesterday Mia had her two-week checkup at the doctor’s office. Her 2-week measurements were 10lbs and 21.5 inches. As you may recall, she was 19 inches long at birth.

If she keeps growing at this rate, she’ll be nearly 6 feet tall by her first birthday. I guess there’s a CHANCE the original hospital measurement wasn’t accurate These measurements basically put her at 90-95th percentile on both height and weight.

Evan is still pretty fond of her. He’ll sometimes call her “Mia Mia”. Of course, maybe he just speaks Italian…