Oral nystatin is what they gave my kids when they had thrush. I've seen it prescribed for 2 week olds so I would say it's safe. You can run your toothbrush through the dishwasher to sterilize it. Also, drink through straws. I know when my kids had it, the pediatrician wanted me to change bottle nipples at each feeding. (The yeast can survive the fridge and rewarming)
Nystatin is an anti fungal not an antibiotic. Its safe to use during pregnancy or doctors wouldn't prescribe it. If you are really opposed to taking it you could try eating yogurt 3-4 times a day. The good bacteria in yogurt can kill off the yeast but I don't know how successful it would be to completely get rid of it. Also either boil your tooth brush or toss it because the yeast could live on it.