What is interesting thing to me about he's older brother is, that I never really had a behavior problem with him. but his younger brother is temper tantrums,cussing and hitting himself and teasing at home and school and most of the time he is anti-social until he feel's like speaking to you or being near you.
The only way is be sure is to get professional help. If there is the chance that he is autistic - the sooner it can be dealt with - the better. There are other reasons for behavioral problems (like ADHD), which need to be considered. This is not something you can do by yourself. The fact that his brother has problems and you have concerns is more than enough to justify your need for professional help.