Dog started growling at people

I have a 4 year old mutt. He migth be a boxer/lab/pitbull/shepherd who knows mix. He has always had issues with other dogs. We work on this by separate feeding, keeping toys and bones away unless supervised. I know his body language and can correct him or the other dog when they start to be posessive of something or someone.  The dog aggression escalates in the winter when I dont exercise him as much as a should.

He has never been aggressive towards humans. He will bark and growl at someone at the door, but once we let them in- hes fine.  HOwever, recently.. he has started growling at people. Just a handful of times.. For example.. I take him on a walk and a man walks by- he growls. We left him in the truck to run in and get a coffee- he growled at someone walking near truck. Last night, some friends came over and he growled at the female friend. I put him in the bedroom and brought him back out later. He was very friendly towards the male visitor- tail wagging, etc. When the female visitor pet him.. he looked very scared- tail tucked. He immediately walked away from her.. sat down and growled at her. I don't know what is causing this.. or what to do. I can handle the dog/dog aggression but not people aggression- this has to stop before it gets any worse.