Their first full-length, Like the Exorcist But More Breakdancing, was released on Eyeball in August 2002. Touring early on with bands like Cursive, Interpol, and the American Analog Set, the band did a summer 2003 split with Volta Do Mar before following up that fall with the well-received Who Will Survive and What Will Be Left of Them? The record was something of a concept album based around the Devil waging war on a small Western town. A benefit 7" appeared at the year's end in memory of Ten Grand's Matt Davis, and the band continued to turn heads nationwide on tours with the likes of Lucero, the Weakerthans, William Elliott Whitmore, and Rasputina.
Edwards amicably parted ways with the group in mid-2004 to go back to school, and when his touring replacement also left by the year's end, the remaining members of Murder by Death decided to just carry on as a more rock-oriented quartet. Balliet, though, picked up keyboard parts along with her cello. After extensive touring, the bandmembers went back to college for a bit while writing their third album. Murder by Death eventually issued In Bocca al Lupo on their East West imprint, Tent Show Records, in May 2006. Another concept album, this one was produced by J. Robbins (Against Me!, Dismemberment Plan), and explored themes of sin and redemption, somewhat inspired by Dante Alighieri's The Divine Comedy. Touring continued through the summer, including a headlining stint with Langhorne Slim opening.
In March 2008, Murder by Death released Red of Tooth and Claw on Vagrant Records. The band followed up again in 2010 with its fifth studio album, Good Morning, Magpie. Two years later Murder by Death returned with Bitter Drink, Bitter Moon, which found the band taking part in a crowd-sourcing campaign through Kickstarter for the vinyl release of the album. The campaign would find them becoming the third highest-earning band since the site launched.