Lyric discussion by TheModNymph 

Cover art for Ashes lyrics by Rufus Wainwright

I think he is describing leaving a lover while he refuses to deny their past's passion.