He is feeling sad and needs more attention,he was a lot younger when the 4 year old was born and was still getting the attention a 2 year old gets but now hes the oldest and he sees thenew Baby getting a lot of Love and if you tell him off especially over and over, it hurts and he feels unloved.Can you not sit him in a chair under supervision and let him have the Baby on his knee, also let him play with the little one soon the baby will be holding Rattles and little toys, teach him how to sing to Baby and show him Favorite Books Babys love any attention at any age and that is good for them aswell.Make him part of you guys and the Baby ,give him little jobs and make him feel he is helping not Hindering, get him his self esteem back.
Although he is in a group program at school, I would still seek counseling to be sure he is getting the help he needs (school may not go deep enough or give him the attention he needs.) My child went through a stage like this. I remember trying to give him more individualized attention, and therapy. You may try asking him to help with the baby. He may just be confused with where he fits with this new change.
YES, seek counceling. The baby may have exasperated things, but the underlying behaviors are neither normal nor healthy. You could also speak to the group program at school about specific things he does at home. Hopefully they can refer you to someone and give more insight.