In the beginning they were good friends ("You used to admire"). They played with the idea of being more ("we fucked around"), but decided against it. He later realizes that he DOES want more, but she doesn't ("But you're not listening"). Now things just feel wrong. He wants to be more for her than she wants ("I always wanted to be The one you looked to").
My interpretation:
Their relationship is constantly changing.
In the beginning they were good friends ("You used to admire"). They played with the idea of being more ("we fucked around"), but decided against it. He later realizes that he DOES want more, but she doesn't ("But you're not listening"). Now things just feel wrong. He wants to be more for her than she wants ("I always wanted to be The one you looked to").