This may be a case of contact dermatitis. As you have noted this to be associated with your wearing the slacks. I suggest that you refrain from wearing tight pants and try to wear loose ones which would not be rubbing on the area. Wash the skin with warm water and keep it dry and clean most times. You may apply 1% hydrocortisone cream once daily for 3 days. If this does not improve, then have this further evaluated.
In the absence of pain, swelling and other associated symptoms, this may not be something serious.