There is something called prostate stones, or prostate calculi, which are really just tiny little stones that can form in the prostate. Most men get these and have no symptoms, but they can become infected, and give symptoms similar to what you are experiencing.
This isn't associated with prostate cancer in any way.
It doesn't just go away though. You'll need to see your doctor to be sure it's this, and get treatment, and to make sure it's not anything more serious. This is not a diagnosis - only a doctor (and I am not one) who examines you can make a diagnosis. Pain is always an indicator that something is wrong, and worth a trip to the doctor. Just wear your mask when you go, and it should be fine.
Let us know what happens.