This sounds like it could be tonsil stones if you are able to remove them and they are hard, as you have described. Although this seems to be a minor case scenario, you'll be fine. No need to see a dentist. Visit your doctor and talk about it. You may have to get an oral irrigator or simply scoop them out with a disinfected oral utensil. Either way, talk to your doctor first.
That sounds limited to your throat. I would see a dentist as a last resort.