Yes you are fine to be in the room. It is not the smell that is bad for pregnant woman but the risk of touching their feces when changing the litter. Look it up
I would suggest against it, especially because the dust can go into the air when changing it, and you don't want to breathe that in.
Have you been around cats your whole life? I've already had toxoplasmosis when I was young so I dont have to worry about it now, but you could ask your doctor to test you for the antibodies. If you don't have them stay away from litter, if you do it doesn't matter either way.
I would say probably not the safest. If you can avoid it, I would.