The psychedelic pop band the Happy Bullets was formed by singer/songwriters Jason Roberts and Timothy Ruble in Dallas, TX, in 2003 after the two met that February while working together at an art gallery. They originally planned only to make records, but local airplay encouraged them to organize a performing group, and Roberts taught his wife, Andrea Roberts, to play bass guitar, leading to the band's first gigs in the Dallas/Ft. Worth area. Later recruits were Josh McKibben (guitar, trumpet, vocals, glockenspiel), Rhett Jones (drums), and Kris Youmans (cello). By the spring of 2004, they were playing around Texas and in Oklahoma as they released their first album, Blue Skies and Umbrellas, on the Jeez Louise label. By January 2005, they had finished recording their second album, The Vice and Virtue Ministry, with producer Stuart Sikes. It was given its local release by Undeniable Records on March 29, 2005, and became available nationally on November 1, 2005.