brain trust: email-routing question

[cellio]

Dear brain trust,

On my domain, I have email addresses that collect a local copy (i.e. I can use webmail on my domain to read them) and also forward a copy to my Gmail address. This is particularly helpful for low-volume addresses that I might not otherwise check frequently.

Today somebody with whom I'd been corresponding contacted me via another channel to report that his email was now being rejected -- by Gmail. Sure enough, the copies are sitting in my domain mailbox just fine, but there's no sign of them at Gmail -- not in trash, not in spam, just not there. Gmail seems to have decided to reject them and not even tell me.

I have questions.

1. How do I get Gmail to stop doing that, at all? If email is sent to my Gmail address, especially by my own forwarder!, I want it to show up there. In the spamtrap is fine if Google thinks it is. Silent deletion is Not Ok.

2. If I can't get Gmail to stop doing it, can I get notifications somewhere?

3. I expected the forwarding from my domain to Gmail to be a private matter between those two parties. Why did the Gmail rejection get all the way back to the sender? Why did I not receive a notice of the rejection at my domain address, which is what sent it along to Gmail? Is there something I can do, presumably via CPanel, to intercept rejections by forwarding addresses?

4. Gmail has filters, which can be used to process incoming email in various ways. I've used them to whitelist a few senders that Gmail thinks are spammers that aren't. When in the pipeline do filters get applied? I think it's after this rejection it's doing, since the message goes nowhere that I can see, but I've whitelisted this particular address now in any case.

Comments

[ellenmillion]

I have no idea how to fix this, but sympathize with the hassles. Argh.

[(Anonymous)]

Google has a long history of silently dropping e-mail whenever they think it's "spammy" enough, without rejecting it during the SMTP session (as they should) or bouncing it or, apparently, notifying the recipient in any way.

While this goes against the SMTP standard (see, for example, RFC 5321 sections 6.1 and 6.2), Google (and Gmail) is, in effect, a large enough provider that they can ignore the standard when they feel that doing so is convenient.

They do offer some guidance on what can be done to increase the chances of an email getting through, but if there's anything anyone (including the recipient) can do to actually ensure that messages get delivered, at least I have failed to unearth it.

[madfilkentist]

The one thing I can think of that could legitimately cause silent rejection of normal mail is an error in the authentication protocols SPF, DKIM, or DMARC. They can be set up to say, "If mail which claims to come from my address didn't come from a domain I've authorized, reject it." It could be worth checking if this is happening. It's possible to say "quarantine it" rather than "reject it," which makes it easier to debug the problem.

[(Anonymous)]

Standard email forwarding can be a bit problematic as it is difficult for the receiving side to tell the difference between a forwarded mail and forged headers (generally spam pretending to be forwarded to try and benefit from the reputation of another domain). Use of the DKIM header (cryptographic signature of the email content) can make forwarding easier as it proves that the original domain sent the email in the first place but that is something that is out of your control as the original sender would have had to add that.

You're correct that this filtering happens before your Gmail rules and is outside of your control.

If I remember correctly, you have two options: * Rewrite the from and to (headers and envelope) of the email when it is forwarded (if you have that option available to you) so that the mail is to your gmail address and from your domain. Gmail hopefully then shouldn't see the forwarded email as spoofed. * Use the Gmail 'Check email from other accounts' option where it logs in via POP3 or IMAP to your domain to fetch email. This bypasses the filtering in question (but still does the standard spam filtering where messages show up under Spam).

Hope this helps!

Matt

[rhialto]

Do you have access to the log files from your outgoing mailer? It may well be that google's refusal isn't 100% quiet, but has some error message there. The one I got was fairly non-informative (I don't remember the exact text) and referred to a web page that basically almost accused me of being a spammer or other ne'er-do-well (and which had no help that applied to the situation).

I did however discover that google seems to be much more strict on its IPv6 connections than on IPv4. No idea why. One would think that the new way is more rarely used (and hence abused) than the old way. But the result was that if I forced my mailer to connect to google using IPv4 instead, my mail would arrive. If you happen to use Postfix I can show my config fragments for that (and maybe I can even remember how it works ;-)

That the original author receives the rejection and not you isn't so strange. From a mail system point of view, you're just a forwarder, and error messages make in general more sense if they go to the author than to some random handler in the middle. Think "this mailbox doesn't exist" type of errors...