Lyric discussion by artslut 

Cover art for I'm Not Calling You a Liar lyrics by Florence + the Machine

It's "oh, but for the grace of god go I" not "on"

hmm, i thought the same thing, she doesnt exactly pronounce the 'n'

I'm certain it's "There but for the grace of god go I". Quite a well-known expression, and the only one that makes any sense. http://en.wiktionary.org/wiki/there_but_for_the_grace_of_God_go_I

Also it was mentioned in an interview as "There but for the grace of god go I" .. not "Oh" . http:// www.youtube.com/watch?v=wAQllvThxKY at the end