I think i've written you before so forgive me if I'm repeating myself, lol. I have bipolar also and although I was taken off my depakote I am still taking lithium albeit a very low dose.
I have all the same worries that you mentioned and can't help feeling stressed that the meds will hurt the baby. However I did have a long chat with my psych and she went through all the statistics with me and it really is only a very very small risk and one which I need to take otherwise I risk becoming ill which would be more harmful to the baby.
Feel free to inbox me whenever you need to chat. xx
While I don't take the same meds as you or have the same condition, I know exactly how you feel. I have ulcerative colitis. During my first pregnancy, my gastroenterologist had me stop taking my meds cold turkey. I did sortof okay without them, but not great. Then I miscarried and had a full flare up from all the emotional upheaval, so I went back on my meds.
With this pregnancy, I went off them again at the beginning, but wasn't able to stay very stable. My OB told me to go back on them since it's better for me and the baby for me to be healthy.
Even though I am definitely doing much better on the meds, like you, I still sometimes feel guilty for not being able to manage without them. Hopefully there's no ill effects on the baby for either of us!