This sounds like a bacterial or fungal infection.
Keep the feet clean and dry and wear cotton socks and well fitting shoes.
You can buy an antifungal cream for athletes feet from the pharmacist. But if you can see your doctor, he may prescribe a cream that is antifungal and antibacterial.
Might it be the beginnings of a fungal infection? I don't know, but I recommend you set up an appointment with the podiatrist about this.
Also, you mention your toes are a bit swollen. Is the top of each foot swollen also, or just the toes? What about the ankles?