Even if you are inside it 100% of the time, it requires violence or the threat of violence to defend against trespassers that might just try to violently throw you out.

Violence is an inherent aspect of human affairs, so the question in every case isn’t whether violence is necessary but rather whether it’s justified.