Keep taking the meds. This is going to happen unfortunately. The thing is taking your medicine will only take the edge off by controlling the mania and depression. Other characteristics of our personalities can hinder too. Things like phobias, codependent behaviors, dysfunctional thought patterns are things that need to be practiced and focused on if you have an issue with any of them.
Any person can react to negative interactions. It would be worthwhile to discuss these concerns with the person who may have caused them. However when a person has bipolar they may sometimes react more to these types of events or at times take them out of context. This has happened to me. Sometimes its best to take some time to think things over first.