Lyric discussion by the_boatman 

Cover art for We Float lyrics by PJ Harvey

I've read somewhere that this is about her relationship with Nick Cave.

He says this about his song Brompton Oratory:

" ... But it's supposed to be where Christ, after the Resurrection, returns to his friends and shows himself to his disciples. I guess I was making a point in that song that some people have it lucky that people do return to them, and in my situation, it didn't seem like that was happening; in my ... in the relationship that I'm talking about within that song it doesn't seem like that was going to happen."

So, the person who left him had done so about a month prior to the pentacost, which would have been Good Friday:

"Die on Good Friday? While holding each other tight "