"Little town" is a metaphore for the relation they are having: a town of two. They don't speak with each other, except to argue and fight. The singer is considering if he should finish his wreck of a relation, decides to throw it all away but, and I am guessing here, is perhaps also uncertain if he could ever get over her? (But then a little voice inside me said, You'll never get away from here)
"Little town" is a metaphore for the relation they are having: a town of two. They don't speak with each other, except to argue and fight. The singer is considering if he should finish his wreck of a relation, decides to throw it all away but, and I am guessing here, is perhaps also uncertain if he could ever get over her? (But then a little voice inside me said, You'll never get away from here)