Lyric discussion by RetiredExplorer 

Cover art for Red Dress lyrics by Motion City Soundtrack

Red dresses are commonly used and have historically been used in poetry and music as a symbol for prostitution. With that in mind, look at the song again. Although Justin is not singing about a prostitute, he is clearly singing about how he enjoys the sexual aspect of a relationship more than the emotional. i.e. "with tilted head like a listener but i'm just thinking about the red dress"

he's pretending to be listening to the girl, but in reality he is just been thinking about sex.