He says something he knows he shouldn't have said - he tried to keep it in, but inevitably it came out and upset all sorts of people. He is now somewhat relieved because he doesn't have to wonder nervously about whether or not his words reached the person they were directed at, but at the same time, he's ready to start his car and run away from the icy stares of the citizens of Fire Island. They are prying at him, trying to get him to say more of what he shouldn't say.
He says something he knows he shouldn't have said - he tried to keep it in, but inevitably it came out and upset all sorts of people. He is now somewhat relieved because he doesn't have to wonder nervously about whether or not his words reached the person they were directed at, but at the same time, he's ready to start his car and run away from the icy stares of the citizens of Fire Island. They are prying at him, trying to get him to say more of what he shouldn't say.