Lyric discussion by hemlock_echo 

Cover art for Casanova, Baby! lyrics by Gaslight Anthem, The

There's a famous quote that's been attributed to everyone from John Cage to Steve Martin that I think is being referenced here: "Writing [or sometimes "talking"] about music is like dancing about architecture" - meaning that it is ridiculous and basically impossible to adequately express one medium in another. I think that the start of the chorus is a reference to this saying. "We could run all night \ and dance upon the architecture" can easily be interpreted as "We can listen to music [Springsteen is specifically suggested] and analyze it". The rest of the song is full of allusions to other songs and the bridge reveals that analyzing the songs is ultimately futile ("And you hear me late at night pickin' at the pieces / Sometimes nothing comes out right") and unfulfilling ("and nothing is relieving").