Lyric discussion by CourageStrengthPride 

Before I read the interview with Cave as to the songs meaning, this is how it always struck me:

The narrator's wife has newly converted to Christianity (or any religion you care to mention), explained in the line "with brand new eyes open wide"; she's looking at the world with new eyes. "Reborn", one might say.

The next verse is her explaining all of the things to Cave/The Narrator God has created, and how 'lovely' and 'amazing' it (her God's creation) is.

The next small part is just her waiting for the rebuttal which will surely come from her grumpy husband (the hair-brushing gesture is quite intimate, I imagine she'd know him quite well).

Verse 4 is, of course, Cave's rebuttal; the world is not as beautiful a place as you think. Look at the people crying out for help and simply being ignored by those who COULD help. We're selfish animals, and even IF we notice the plight of others we're not likely to stop to help.

Next verse is just him brushing her hair from her eyes and the kitten moving back to her lap (it's like a hairy talking stick :P).

The last verse is the 'victory'; his wife loses her temper at him and gets all haughty. She insults him and the two sit quietly. He knows now that the religion is just an addition to her life, she's still the woman he married and, consequently, loved.

The refrain ("As I sat sadly by her side") would just be his sadness at her 'succumbing' to religion.

That is, this is what I thought until the interview. Close enough.