It can be due to dyshidrotic eczema or contact dermatitis. Wash the areas several times with fresh water. Do not use any cosmetic products at the sites. You can apply some calamine lotion at the rash as it will help in soothing the skin. You can take some OTC oral antihistamine medications like Benadryl or Claritin and see if it helps.
For mild-moderate symptoms a weak steroid may be used, whilst more severe cases require a higher-potency steroid.But they are available under prescription and should be taken only after consulting a dermatologist. Symptoms can be exacerbated by dryness of the skin. Use good quality moisturizers to prevent moisture loss from the skin.
If they prove unsucvessful, then Light treatment: UVA-1, PUVA, Grenz rays, Low Level Light Therapy using a Red + NIR (LED) combination can be used.
It is very difficult to precisely confirm a diagnosis without examination and investigations and the answer is based on the medical information provided. For exact diagnosis, you are requested to consult your doctor. I sincerely hope that helps. Take care and please do keep me posted on how you are doing.