Lyric discussion by Seannn8706 

Cover art for I'll Let You Live lyrics by Taking Back Sunday

it sounds to me like he could have found a girl who was with a good friend of his and kind of took her from him. He feels guilty by messing around with a good friend's g/f but the two of them are so infatuated they can't hold themselves back

He wants to feel guilty and end it but in being honest with himself he knows he's afraid that this girl will leave him.