What I'm reading from the song is it seems like a song after J started putting his life together. Some of his less than scrupulous friends continue to call him and ask him to go get high with them. And he declines, and that itself is fine for the thought of asking. However, these friend/s it seems are drug fiends who aren't doing anything with their life other than getting high. It's almost as that is all they're doing. So, getting high with them would also be counterproductive to what he is currently trying to accomplish.
What I'm reading from the song is it seems like a song after J started putting his life together. Some of his less than scrupulous friends continue to call him and ask him to go get high with them. And he declines, and that itself is fine for the thought of asking. However, these friend/s it seems are drug fiends who aren't doing anything with their life other than getting high. It's almost as that is all they're doing. So, getting high with them would also be counterproductive to what he is currently trying to accomplish.