Words are not violence, neither is silence, despite the rhyme.

Yup they are not, but sometimes they are a justifiable excuse/reason. But I will point out that if the person that starts the verbal altercation generally in my eyes loses virtually all of the justification because they started it.

sometimes being the "bigger man" and walking away just puts a big fucking "kick me sign" over your head.

Maybe but I tend to think the message is more something along the lines of "you are so far beneath me it's not worth the trouble to awknoledge your existence".

It is always situational, if its a one off interaction? Then yeah trying to disengage is probably the best response, but of its a pattern of behaviours that will continue that puts it in a different light. At lest in my eyes.