Lyric discussion by KatherineKissMe 

Cover art for Ghosts lyrics by Florence + the Machine

"I'm not calling you a liar, just don't lie to me/I'm not calling you a thief, just don't steal from me..." - She doesn't care what her lover has done, just as long as he doesn't do them to her, and she doesn't want to know; listen to the original version of this, Don't Tell Me, "Don't tell me what you've done." Pretty obvious really. "I'm not calling you a ghost, just stop haunting me" - The 'haunting' says to me that she can't get away from him, whether it's his intentions or her not being able to let go. "There's a ghost in my lung and it sighs in my sleep..." - He is controlling her (again, whether it is his intention or hers isn't clear), and whatever her does leads her to fall at his feet. "There but for the grace of God go I" - This is a common phrase that basically means that somebody elses misfortunes could've just as easily happened to you

My Interpretation