Lyric discussion by Rozhestvensky 

Cover art for Two Swords lyrics by English Beat, The

Non violence isn't the idea here, it's saying the guys who hate nazis are probably nazis themselves, by fighting them we become no better than they are, maybe worse.

@Rozhestvensky I disagree ... no where in the song does it imply that they are a Nazi for hating Nazis.