Lyric discussion by imfloatingaway 

Cover art for Sweet Talk, Sweet Talk lyrics by New Pornographers, The

Does anyone know what it means he talks about feeling "byzantine"? I can't figure it out.

Okay, I'm totally pulling this out of my ass here, but I think he's using the metaphor of walking down the street in your Sunday suit (i.e., your nice clothes) for spending more time dwelling on something (the "mistake on the part of nature" that's been forgiven) to mean that both are excessive and unnecessary. I don't know though, that's just my guess.