Lyric discussion by xstartstopx 

Cover art for Numbers lyrics by These New Puritans

I think they're talking about how symbology can be as simple as a number.

"What's your favourite number, what does it mean?"

Something as simple as that can mean so much to certain people, symbols mean a lot to the beholder, even if it doesn't mean much to someone else. "Every number has a meaning" (Everything has a meaning)