Yes, it is. everywhere I've looked it says it most definitely is, and recommend that you get someone else to do it for you, or make kitty go outside for potty time
The smell is not harmful. It may make you lightheaded but its not going to harm the baby. Smell is a sense, baby cannot smell what you smell. Touching the litter however can be dangerous due to the risk for toxoplasmosis. If you have to scoop the box, wear gloves and wash your hands thoroughly. I have 4 cats and while my hubby usually does the boxes, he was away on business a couple months ago and I had litter duty for 3 weeks. You'll be fine as long as you're smart about it.
As long as you don't touch it, it won't cause any problem for baby. Personally, I use natural pine litter, because there's no clay dust floating around while changing the box, and it's way lighter and easier to dispose of, so if you're stuck on cat box duty, I'd definitely try the pine litter. It makes my life a lot easier! Lol