Lyric discussion by terrycraig 

Cover art for Brompton Oratory lyrics by Nick Cave and the Bad Seeds

The third verse is very likely a reference to The Hunchback of Notre Dame; The classic 1939 film version with Charles Laughton ends with Quasimodo leaning on one of Notre Dame's stone gargoyle statues, bitterly saying "Why was I not made of stone like thee?" while seeing the beautiful Esmeralda go off with somebody else.