not the customer but the product

[cellio]

There's apparently another widespread Gmail outage, but this one is more harmful -- it's lying to senders about addresses being invalid (permanent error).

This might be the swift kick in the rear that I needed to figure out a different approach to email. I have a domain, so I should set up a single "collector" address there to receive everything I'm currently forwarding to Gmail (which I'll have to hunt around for; Pobox is easy but not the only one). I hadn't done that before because I thought that relying on Google (a huge, hardened service) was a safer bet than relying on my domain -- what happens if my domain gets hijacked, my hosting company compromised, etc? Rethinking that now...

Fortunately, I'm already forwarding Pobox to an address on my domain, a backup for Gmail, so I probably haven't lost anything. But I might be getting silently dropped from mailing lists I cared about. We'll see.

---

Ok, I think I now have everything going to one mailbox on my domain and, from there, mirrored to Gmail for now. I'd like to have all my mail in one place, but the last download of my Gmail mailbox was a 10G file in mbox format, which I don't know how to read or plug in to something else. (I mean, obviously that's a standard format, but what can I use on my Mac to read it?) I don't really want to store all that on my domain server long-term (it'd raise my storage costs), but there's probably a lot of junk in it, mixed in with the stuff I care about. I'd already done some passes to, for example, nuke years-old mailing-list threads that I don't care about now, because Google has storage limits, but that's time-consuming.

I welcome input from people who've wrangled large mailboxes, domains, and email more generally.

Comments

[(Anonymous)]

There should be an import path from mbox into just about anything these days (though you might need to go through an interim format). It's really just a flat file with every message concatenated, separated by a line beginning with From ... (the space separates it from the From: ... header). It can be wrangled into one-file-per-message quite easily in a variety of ways, everything from specialized tools like formail to homemade scripts.

Mutt works fine, especially if you don't mind a text-based user interface, and is extremely configurable, but I'm not sure how well it'll handle a 10 GB mailbox as-is. If you go that route it'll absolutely have to be on extremely fast storage; spinning rust won't do. (With mbox, every change requires rewriting the whole file.) Header caching will help, and Mutt has native support for that, but splitting the mailbox up will help more, ESPECIALLY if you're using mbox format for storage. Maildir would be more appropriate, but you'll have to set up some kind of header caching in that case for the UX to be bearable. No matter how you slice it, 10 GB is a fair amount of data.

What Mutt lacks is easy support for non-plain-text main portions of mail. (You can wrangle it in, but I'm pretty sure it won't do anything like that natively. It can be set up to render HTML mail, but unless you launch into a separate application, it'll be a plain-text rendering. Think Lynx. Really, Mutt is little more than a specialized text reader; admittedly, one very good at what it is supposed to do.)

As for GUI-based alternatives, I'm pretty sure Thunderbird can import directly from mbox.

You could also run a local IMAP server like rhialto mentioned, point your various devices to it and set them up to leave all mail on the server, but then you'll need something that downloads your e-mail and passes it on to that IMAP server, and of course you'll need somewhere to run the IMAP server, ideally not bound to your main computer at home. Certainly doable (the Mail-in-a-Box project looks interesting), but probably worth paying for the service of having someone else do it for you if you don't have another reason for wanting e-mail fully under your control (and are willing to put in the time to set up and maintain that). Plenty of providers offer decent storage space for e-mail and give you IMAP4 access at a reasonable cost, and many also offer outgoing SMTP and webmail access, which sounds like it'll have you covered.

If you want to run something of your own, plenty of VPS providers will give you 20-40 GB of storage, a reasonable CPU and RAM allotment, IPv4 and often also IPv6, for a few dollars per month.

Running your own outgoing SMTP server these days is fraught with issues. (Not thank you, spammers!) Frankly, it's not worth the hassle to try to get it to work reliably. It's better to pay some company for the service of delivering outgoing mail, and let them deal with the headaches.

Says the Dog :)