I can understand your nervousness. It's not just about how you were treated or will be treated in your remaining appointments, but how his lack of care may reflect how the birthing will go. My wife and I have talked to numerous friends about the differences between doctors and midwives. The biggest and most consistent thing they all pointed out is that doctors generally want to rush things and get it over with. They tend to want to drug you to force body relaxation, and when the baby doesn't want to come out, they always push for a c section.
My coworker today talked about that happening to his wife for their first birth. They managed to push back to not get a c section, but the birth lasted 72 hours due to all the drugs. Their second birth was at home with a midwife, a completely different experience he said. Baby came out just like that.
My wife and I are going with a midwife and we never had to wait more than 10 minutes for initial appointments. Now, they come to our house every time, with no wait!! If you are feeling unhappy or nervous, maybe you can ask around to see if there are other doctors or possible midwives. Best of luck to you and your baby!
It's totally acceptable. If you're just stuck on the time constraint I'd find a midwife. You'd still have an OB deliver at the hospital, but midwife can voice your plans if you'd like certain things during labor and delivery. Like natural birth.
What exactly are you expecting from your doctor at your last remaining appointments?
What do you want specifically, besides not to have your time wasted waiting for your appointment?
I think that's what you need to focus on. What you would ask of a different doctor that you aren't getting now, besides time. Do you have questions you need answered? Do you want tests run that aren't being done?
Before you switch I think you should have pretty clear requests of the new doctor, so that you don't come off as just wanting someone to talk to.
Best wishes.