Lyric discussion by paul_f 

Cover art for Beyond Belief lyrics by Elvis Costello

A gin palace is a particular kind of ornate Victorian pub of which some still survive. There are a few great examples in Liverpool where EC spent a couple of teenage years. Bone orchard is a common term for a cemetery in Liverpool (and elsewhere I believe).