"The song uses piano to swing in what Darnielle termed an "Italian cafè style." It is something of a sister song to "In Memory of Satan" in that it looks at a couple on an antisocial downward spiral, although they are trying to move forward. "You look at this one as what ‘Satan' would be with more than one person involved. It gets into the coping strategies you use for mental illness, which are really the fabric of what keeps society moving forward at all," explained Darnielle."
I read an article that said:
"The song uses piano to swing in what Darnielle termed an "Italian cafè style." It is something of a sister song to "In Memory of Satan" in that it looks at a couple on an antisocial downward spiral, although they are trying to move forward. "You look at this one as what ‘Satan' would be with more than one person involved. It gets into the coping strategies you use for mental illness, which are really the fabric of what keeps society moving forward at all," explained Darnielle."
More here: http://www.rollingstone.com/music/news/mountain-goats-john-darnielle-discusses-the-satan-record-20120120
In addition, I think the cave mentioned in the song might be Plato's famous, metaphorical cave.