Lyric discussion by Blithium 

Cover art for Genesis 3:23 lyrics by Mountain Goats, The

I'm stricken by the fact that the narrator had to break the lock on HIS garden gate when he got home. Why didn't he have a key?

I think it's more like he's breaking the lock as a big "Screw You" to the Big Man Upsairs. If the character in this song is/resembles the mental state of Adam post-banishment, then it makes sense that he wouldn't necessarily care for locks on garden gates.

It's a way of saying, "I'll be more welcoming and understanding than You."