The 'wish you would leave me' etc. always implies to me a self-loathing, or a suicidal painedness where there's this one good thing left; dazzlingly good but the rest is awful. 'you' leaving 'me' would remove the justification, which could be easier.
And I'm agreeing about his voice: things like "I just can't go on" are not good lyrics without his yell.
The 'wish you would leave me' etc. always implies to me a self-loathing, or a suicidal painedness where there's this one good thing left; dazzlingly good but the rest is awful. 'you' leaving 'me' would remove the justification, which could be easier. And I'm agreeing about his voice: things like "I just can't go on" are not good lyrics without his yell.