Trust your instincts but don't fret, the truth always comes out. Just focus on you and the baby and let him know how you feel. Stress is not good for you or the baby, it may be just as he said.
I dont think your hormones are seeing things, esp to hide the bag behind the house.... something is going on maybe a minor thing but still he should be honest!
Maybe something is up i don't want to be the barrier of bad news but maybe he is cheating