Converting from Netscape or Mozilla to IMAP

This page discusses some of the fun I've had trying to convert my 400 megabytes of e-mail from Netscape's internal format to something digestible by the University of Washington IMAP server. The storage format for Mozilla and Netscape Communicator are largely the same, so you should read this as applying to both systems. Where differences occur, they're noted.

Why bother?

I run Netscape Communicator as my mail client. It's got a whizzy interface. It supports HTML and S/MIME. Good enough, but when I traveled, I couldn't see all my saved e-mail. With POP, all I could do was set my laptop to leave mail on server, and clean up the mess when I got back to the office.

IMAP has been touted as the solution to the problem. No matter where you go, an IMAP server provides your e-mail to you. You get to work off-line and synchronize your changes when you next reconnect (including refiling and deleting of messages). I wanted to do it.

So what's the problem?

Despite the fact that both Netscape and the UW server support the Berkeley mbox file format, both have extended it in various ways that are incompatible. Netscape, when you run it from your PC, will happily write out CR-LF as a line separator. This makes the UW server very unhappy. Also, Netscape stores its status flags (have you forwarded the message to someone else? have you replied to the message?) as bits in a custom X-Mozilla-Status header. The UW server stores these flags as ASCII letters in a new X-Status header. The UW server and Netscape also have different ways of representing sub-folders. Netscape silently appends a .sbd suffix to all folders, which lets Netscape tell the difference between a message folder and a subdirectory. UW has no support for this blindingly simple idea, instead forcing IMAP clients to deal with folders which can hold messages being different from folders which can hold other folders.

Yuck! What's the solution?

If you're running anything other than the UW imapd, I have no idea. Much luck to you.

Otherwise, download the following perl script, nstouwimap, and read the comments in the header. You basically run it with two arguments: the location of your Netscape mail folders and the location of your new IMAP mail folders. Make sure these are in different places! Also, exit your browser before running nstouwimap!

This will grind away, ultimately translating your mail from old format to new. So far so good. Restart Netscape and reconfigure it to use your IMAP server and set your IMAP Server Directory, on the Advanced dialog box tag, to point to it.

When you open the IMAP folder, you should see the same basic folder structure that you had with Netscape, except you'll see some folders replicated twice. One of them has the top-level messages. The other has the sub-folders. Poke around and you may find you need to clean up some of the mess. Odds are, you'll also discover a bunch of annoying client bugs, where, even if you tell Netscape to Synchronize all your folders, it still manages to miss a bunch of them.

Eventually, running overnight in my case, I finally got Netscape to make local cached copies of all my e-mail. Finally, things run fast again, despite burning disk on my PC as well as on the file server. Also, my laptop now also has cached a full copy of my mail folders. This is what IMAP is all about.

Anything else?

You bet. If you had a lot of Netscape mail filtering rules, you're going to want to change them to now refile to your IMAP folders instead of your ``local'' folders. Quit your Netscape again.

Now, when you switched from POP to IMAP, Netscape copied your mail filtering rules into a special IMAP directory, i.e., C:\Program Files\Netscape\Users\default\ImapMail\mail.you.com\rules.dat. The fastest way to fix this is to edit it with Emacs, search for all the actionValue lines and mechanically change the strings. You can replace all the .sbd bits with _ and you can prepend all the target folders with //imap:. (Mozilla uses a more modern imap:// prefix, but otherwise it's the same idea.)

You also might consider changing the location of your Inbox, Trash, and Sent folders to be on the IMAP server instead of on your local mail. That's all stored in Mail & News/Copies and Folders, in your Netscape preferences.

Am I done yet?

Pretty much. I can now see my mail via IMAP with Netscape and using our local Web-to-IMAP gateway (webmail.rice.edu). Communicator 4.6 occasionally seems to drop two copies of a message into the target folder when refiling, but otherwise works. The UW server once got confused and was showing me messages that contained the headers of previous messages, but I haven't been able to reproduce the bug. Outlook98 pretty much chokes; it will show me my Inbox but for every other folder it says there are no items to show in this view. Another quality Microsoft product.

It's not working for me...

There are no warranties expressed or implied. You're dealing with cruddy client software, flakey server software, and a late-night hack to convert your data across.

Acknowledgements

Jon Earle discovered that UW imapd is unhappy when `From' lines are malformed and contributed a patch to fix it.

Dan Wallach, CS Department, Rice University
Last modified: Wed 30-Oct-2002 10:25