I had something like this, near the anus in the crack going in the direction of my back. It was painful and would occasionally rupture(gross I know, but part of the description) The doctor told me that is was a common thing that happens with ingrown hairs especially in that area, and I can either wait it out, or have surgery to permanently remove the hairs. I waited it out, and they occasionally affected me for like 4 years, and then went away for some reason. Maybe this is the case for you.
From your description, this may very well be just a sensitive pimple. Before you go to the doctor, since I see that you are married, get on hands and knees and have your wife closely inspect this anal area under strong lighting She may very well be able to tell you what it is or at least describe it more completely.