I think that Ed's position would be the same if you said "Well you're an idiot!". Idiot is not an offensive word per se. It's the alleged attack.
That's what he's talking about, and I agree with him. Heck, it's not even always clear to us (the not-so-new members) when you're joking or not...