I was told by my Dr to not use icy hot. the methyl salicylate ingredient can be harmful to the baby.
I suggest a warm bath and use Epsom salts. It's a more natural way of relaxing the body. Light some candles and just take a load off.
On your ba k yes, but under your stomach I wouldn't. That will be extremely painful believe it or not (experience here lol) and don't do your butt cheeks either.