Lyric discussion by Gloom 

Cover art for Song of Joy lyrics by Nick Cave and the Bad Seeds

Yeah, I think he was the killer he was talking about, who now just wonders possibly looking to just recreate his crimes. 'In my house he wrote "his red right hand" ' is a line that strongly suggests it was the story teller himself (Cave, in a fantasy role of an insane Cave) who did it.