Yes, the "he" in the song is Me's father.
In this song, Me is just a small child, afraid of anything, trying to strive to become better than his father, [who is later revealed to be worse off as a human being] because Me is inferior to him in every way. The father always takes the opposite end of Me's opinions, always posing as an adversary. During this song it is also revealed how his father thinks ill of him [And then he would tell her / How I tripped and hurt my head, how I’d always be a loser], and doesn't care for him at all. This results in Me's uncaring attitude towards his father to the end of the song.
Yes, the "he" in the song is Me's father. In this song, Me is just a small child, afraid of anything, trying to strive to become better than his father, [who is later revealed to be worse off as a human being] because Me is inferior to him in every way. The father always takes the opposite end of Me's opinions, always posing as an adversary. During this song it is also revealed how his father thinks ill of him [And then he would tell her / How I tripped and hurt my head, how I’d always be a loser], and doesn't care for him at all. This results in Me's uncaring attitude towards his father to the end of the song.