I agree with caregiver. Your pharmacist may help you with drug drug interactions. Sometimes using one drug to treat the symptoms of another drug can be a snowballing problem. I know that Ibuprofen can often give rebound headaches. Try going drugfree and use tylenol for the headaches. At the recommended doses it will be say to use with breast feeding and should not interfere with birth control.
All of this is hearsay, and while I am not contesting your version, I am reluctant to condemn a physician after only hearing one side of the story.
Ulcers are caused by a bacterial infection of helibactor pylori, and have no significant relationship to medications. That statement is qualified, because ulceration can be caused by a pill resting against the lining of the mucosa for an extended period, but this is rare.
If all that you say is correct, that you are breastfeeding and were prescribed medications that are contraindicated when doing so then it would appear reasonable and prudent to immediately switch physicians. Immediately. Discontinuing the meds prescribed should have no significant bad effect. Yes, I would stop treatment.
You are not being overconcerned.