Friends They Are Jewels Lyrics
On the steeple birds and this red-light hotel
The company men never came to you
But don't unknit your brow, Granny
The mice in the yard ate the potted plants you grew
Perfect polished stones but this breeze beats you both
The duty of men never fell to you
When you unknit your brow, Granny
Your friends, they are jewels, twice as beautiful and few

The song is played very slow, helping give feeling to the lyrics showing that life is much more beautiful when we open our minds (lay your pistol down) and that life is short (granny).
Beam also shows the detrimental effects too much caring can have, as if someone had spent so much time polishing stones but forgot to enjoy the breeze, or when spending lots of time growing plants that are just to be eaten by mice. (Perfect polished stones but this breeze beats you both, The mice in the yard ate the potted plants you grew)
As for "The Duty of men never fell to you", I'm not sure but it just might be an accompanying line saying "you've had it easy so far, so why are you even raising your pistol?" in the most lyrically relevant terms possible.
Basically, we should open our minds and enjoy the true beauty in life while we still can, because in the short time we have, there's not much else to do.
That's just my interpretation. I also believe that this is one of my top three Iron & Wine songs ever. It's so beautifully and craftily written.

I think he's talking to an old woman (Granny) about how much more she would enjoy life if she started worrying about the smaller stuff in life. (Lay your pistol down, Granny......... But don't unknit your brow, Granny, the mice in the yard ate the potted plants you grew......... When you unknit your brown, your friends, they are jewels, twice as beauitul and few) This song is wonderful and touching.

so laaay your pistol down , grannyyy...the duty of men never fell to you... I think it's exactly what you want it to be... yes you should embrace the smaller but twice as beautiful things in life...granny before you die

Amazing song.. so much feeling in the guitar.. "your friends they are jewels.. twice as beautiful and few".. Poignant