Even without the government, you need to enforce the system through violence. It's not the non-violence principle, it's the non-aggression principle. Anarcho-Capitalism (which I believe you support?) allows violence so long as it's in self-defense or in defense of one's property. You just can't be the aggressor.