I believe the "Losing end" is referring to being in a one-sided relationship, or a past relationship where one person put way more effort into it than the other.
By "hard to make love pay" I believe he means it's hard to get what he wants out of the relationship because the other person will only put so much effort into it (lack of caring or whatever reason), and even so, he still tries so hard. This in essence is the meaning of "the losing end".
This is how I interprit the song anyway.
I believe the "Losing end" is referring to being in a one-sided relationship, or a past relationship where one person put way more effort into it than the other. By "hard to make love pay" I believe he means it's hard to get what he wants out of the relationship because the other person will only put so much effort into it (lack of caring or whatever reason), and even so, he still tries so hard. This in essence is the meaning of "the losing end". This is how I interprit the song anyway.