Lyric discussion by supposablethumbs 

Cover art for Candy Jail lyrics by Silver Jews, The

I'm thinking this is about rehab. From the perspective of a guy in rehab, who made it to a safe harbor and avoided the fate of other addicts (I came all this way to see your grave...this is what you couldn't face).

"The warden really listens and he understands." Addiction counselors are typically ex-addicts.

The dark side of this song is that everywhere this guy looks, he's thinking about the "candy."