Does he have any developmental issues? My nephew who is above average in school was acting up in school and even got suspended for pushing a kid. It turns out he is frustrated and teased because he daydreams in school ( or so they thought) What the problem really is is that he has an uncommon auditory disorder where he hears but then there is a delay with information that he hears and how it is processed. It is not AHDD but Central Auditory Processing Disorder
Also anything changed in his environment at home or any other changes?
KCSC, I really think that if more than one parent has called you with concerns, and has said they don't want their child around yours, it's more than breaking pencils and playing too roughly. For a parent to screw up the courage to call you and say your child needs to stay away from theirs, the behavior is probably pretty extreme.
I don't think it's video game playing causing this.
When you've observed this behavior, what seems to be causing it? Is he typically bullying kids who are competing with him, or is he more typically bullying very helpless weak children?
What does his teacher say?