Lyric discussion by Xeroxed 

Cover art for Young Lust lyrics by Pink Floyd

I agree entirely with what mahakala said.. I couldn't exactly articulate what I wanted to say, but yes. picking up groupies/"dirty" women is just one way to distract him from the thought of his wife with another man, and a way to keep adding bricks to the wall.