LJ and mail problems

[cellio]

Apparently I'm one of the lucky folks on the MadCow cluster, the one that was broken more badly than all the others. Oh joy. Glad it's back up earlier than predicted, anyway.

To add insult to injury (well, injury to insult is more like it), my mail provider seems to have stopped receiving incoming mail several hours into Shabbat. After Shabbat, when I discovered the problem, I set up pobox.com to mirror to a second address, but I'm missing the mail that arrived up to that point. I hope it's in a queue somewhere and not gone forever. I also hope they respond to my report soon; while I realize it's Sunday, it's been (at this writing) 16 hours since I made the report and well more than 30 hours since the failure occurred.

A service that I would really like to have, but lack the technical skills to implement myself, is an inbox buffer. The problem with mirroring the pobox redirection to the mailbox that comes with my DSL is that if I don't remember to clean it out periodically, it fills up and starts bouncing. I could use gmail on the theory that a gig is forever, but the vast majority of my mail is spam and I don't want it getting in the way of my real mail if I actually want to use that service for searching. So what I'd really like to have is an email address that receives mail and automatically deletes it a week later; that gives me a backup that I can check if for any reason my primary email is unavailable, but it runs on auto-pilot the rest of the time.

Vaguely on the subject of mail, I got (physical) mail this week from the ACLU telling me it's time to renew my membership. Just one problem: I'm not a member of the ACLU. I really expected better behavior out of folks who claim to be fighting the good fight. Deceptive marketing practices do not endear them to me.

Comments

[tangerinpenguin.livejournal.com]

I get very, very little spam at my gmail address, and what I do get is autofiled pretty effectively by their filter (in fact, I've had to "rescue" something I'd actually opted into once or twice.)

[cellio]

I get lots of spam at my pobox address, which I would be redirecting to gmail in this scheme. But I'm glad to hear that their filter is pretty good; that does help. (Since I haven't published my gmail address, I haven't paid a lot of attention to the incoming feed. My plan is to bounce all my archived mail there -- in my copious free time, of course -- to use the search features, not to use it as a "live" address.)

[siderea]

Um, I'm confused about what you want. It is possible that if I understood it, I could make it happen for you.

To have an email address which receives a cc and automatically deletes it, one needs two things. The first is a way to split your mail stream and send a copy of everything to the backup site. It sounds like pobox.com does this for you?

The second thing you need is a target account with an automated, asynchronous deletion thingy. A shell account with procmail will do that for you handily. The part I am unclear on is whether you have such an account available for you already, or whether you're willing to pay for such a thing.

If you have such a resource, I can go kick around and generate the procmailrc to do what you want.

The question is just where you want your point of failure to be (speaking as someone who has done a lot of just this sort of thing, for just this sort of reason.) There is always a single point of failure with an email address. That's just how it is. It is possible to have the procmailrc live on the same server (or even be the same script as) the mail splitter. But that might not move your single point of failure where you want it.

[cellio]

Thanks for the offer of help.

As you inferred, pobox.com takes care of the splitting. Mail comes in and will go out to as many addresses as I set. (There's no partitioning; all the mail goes to every address.)

What I'd like is to set up a mailbox -- somewhere -- to receive pobox mail, such that mail goes in and gets auto-deleted a week later (to avoid filling up the disk). Most of the time I wouldn't access it, but if there were a failure at the primary mail drop I could at least make sure I get all my mail.

I do not currently have such an account. I have an email address through my DSL provider, but no shell account to go with it. On the other hand, I access that mail using pine on my windows box at home, so if there's a way to set up the scripting that way, that'd be fine. I have no problem leaving my home computer logged in to my DSL mailbox.

I also have a gmail account, but so far as I know they don't allow you to do anything fancy.

I've just realized that I have a friend with a mail server who has provided accounts for some of our friends. I will ask him about the details and, in particular, if he runs procmail. If so, that'd probably be very cheap or free. Failing that, I'm willing to pay for this service but would of course like to keep the costs down if I can; I'm not sure what the optiosn are for seldom-accessed email/shell.

I realize that pobox is currently a single point of failure; if it goes down then no mail flows. I'm counting on the large customer base and the fact that they've been reliable for the seven or eight years I've used them.


Grr. We are now at 40+ hours since failure and 22+ hours since my report, and it appears that they bounce failing mail after one day. (My test messages to myself have generated "will try for one day" soft bounces.) So most of a day's worth of my mail is apparently just plain gone. I'm getting a little testy that I haven't seen any movement on this at all. Is this typical these days?

[siderea]

Aha! OK.

The ISP I use, nyip.net, has a $54/year model that will do you, I think. It has only 350MB space; would that be an adequately large buffer?

Regardless of where you go, I will go cook up a procmail script for you. (I think there's pretty much an example in the man page that will do the trick.)

What format mailbox would you like your mail stored in in your backup system? Are you familiar with the Unix flavors of mail box?

[siderea]

OK, the man page example is a backup of the last N msgs on a rolling basis. Would that be OK, or do you want it by last N days?

[cellio]

The ISP I use, nyip.net, has a $54/year model that will do you, I think. It has only 350MB space; would that be an adequately large buffer?

Thanks for the tip. 350MB should do the trick; it's certainly a multi-day buffer, so if that turns out to be 5 or 6 instead of 7 that's still ok. If my friend (to whom I've just sent email) can't sell me an address for less, I'll sign up with them.

Thanks very much for the procmail-wrangling! I currently use a very very simple procmailrc, because I don't really know what I'm doing. (Mail from one high-volume mailing list goes into its own folder, anything with a SpamAssassin score of 5 or more goes into its own folder, and the rest goes to the inbox.)

What format mailbox would you like your mail stored in in your backup system? Are you familiar with the Unix flavors of mail box?

The answer to the second question is "no", and thus the answer to the first one is "I have insufficient clues to answer that". What are the relevant factors?

The inbox for my current primary mailbox is a text file in /var/mail; I don't see any special sequences at the beginning of it, but if you tell me what to look for I could try to figure out what it is. Pine knows how to parse it, and that's been good enough for me. :-)

[cellio]

Last N messages should work; I'll just have to do some metrics to figure out what N is. :-) (Actually, I think I can get that data from pobox.)

[siderea]

The answer to the second question is "no", and thus the answer to the first one is "I have insufficient clues to answer that". What are the relevant factors?


The inbox for my current primary mailbox is a text file in /var/mail; I don't see any special sequences at the beginning of it, but if you tell me what to look for I could try to figure out what it is. Pine knows how to parse it, and that's been good enough for me. :-)

Ah, you want a standard spool. No problem. If you have a file with some other name, you can get pine to read it? Or get it to where pine can read it?

[cellio]

The pinerc file has a parameter for specifying where the inbox is if it's non-standard, so the name/location of the file shouldn't matter.

[siderea]

(Now, with correct formatting!)

It turns out that by day works better with standard spools. :) Righto. I think this will do what you want, but you want to test it before relying on it:

1) Make a directory called "Mail" or whathaveyou, which shall be henceforth referred to as "$MAILDIR"

2) In $MAILDIR, make a directory called "Backup".

3) In the root of your homedir (or wherever your sysadmin tells you is the place procmail will look for recipes) put the following into a file named .procmailrc:

:0 c
$MAILDIR/Backup/spool_`date +%F`

:0 i
| cd $MAILDIR/Backup && rm -rf dummy `ls -t spool_* | sed -e 1,14d`

The "1,14d" in that last line is what tells is how many days worth of spools to keep around; it's specifying 14, salt to taste. This recipe should result in a a different spool for each day, with all the day's mail going into a spool called $MAILDIR/Backup/spool_$yyyy-$mm-$dd. It keeps the last 14 of these around on a rolling basis, checking with every email that comes in.

No, I haven't tested it. :) It is based on the man page example.

[geekosaur]

I hope it's not Panix, whose domain an unscrupulous registrar allowed to be stolen (no, it didn't expire; they simply gave the domain to someone else without authorization and without telling anyone) and is now being decidedly unhelpful about setting things to rights. :( Speaking of deceptive....

[cellio]

Not Panix (but that sounds really evil!), and the outage appears to be specific to mail. They're still serving web pages and I can reach my shell account through SSH.

They just sent email saying "we know about the serious email problem and are working on it". Of course, only people using external addresses will see that mail. A notice on the bug-report web page or the login greeting would have helped, but I guess they didn't think of it.

[goldsquare.livejournal.com]

The ACLU has, apparently, also been doing background investigations into their donors attempting to find out more information about them and finding relatives that might also wish to contribute.

They still do good work, but I'm not so happy at how they do it.

[cellio]

Thanks very much! As soon as I have the backup shell account set up I'll give this a spin.

[madfilkentist]

The AAA sent me a bogus expiration notice this month.

[cellio]

How odd. I've been a member of AAA for close to 20 years and I've never had that experience with them. Is there a possibility of clerical error? (e.g. did you just move or do something else that would affect your record with them?)

[cellio]

Feh. That information further lowers the likelihood of my giving them money. (I wonder if one of my relatives unknowingly set me up.)

[cellio]

My friend was able to give me a shell account. And, serendipitously, he uses his for the exact same purpose and already has scripts in place to auto-delete mail after a week, so he set me up with it. Score! I've now hooked that account up to pobox and am waiting to see what happens in several days.