Lyric discussion by Kangus 

Cover art for My God Is the Sun lyrics by Queens of the Stone Age

I would think it has to do with ignorance. A lot of primitive cultures worshipped natural phenomena as the work of a deity, in this case, the sun. Some other lyrics also support this, such as the fact that the singer doesn't wear a watch (as in he doesn't know, doesn't care). Also, the insignificance of being an ant (atop a spinning rock [AKA the Earth]). Another nod to primitive cultures is the whole waltzing in/on the sky, as early cultures such as the Greeks thought that when heroes died, they became constellations. In that case, the speaker is probably someone else, as he is calling those people heathens.