offensive or over-sensitive: how does a moderator judge? (can you help?)

I moderate a few online communities, and occasionally something gets flagged as offensive that doesn't strike me that way -- but, in a large heterogeneous community, it can be hard to know whether I don't see it because it's not there or because my own perspective blocks it. Put another way, is that my privilege speaking?

Today I decided to ask that question on the fledgling community-building site on Stack Exchange. If you have experience mediating such issues, please consider answering there. You could comment here too, of course, and I'd like to hear what y'all have to say about this, but I hope that if you can speak from actual experience you'll consider sharing your knowledge over there where it will help people other than me too.

I'm oft fascinated by how online communities work (or, sometimes, don't work) and I'd like this site to succeed. Also, I've written some good stuff for it that I'd hate to have disappear from the network if the site doesn't reach critical mass.