When was the last time he had a physical? The doctors usually put their height and weight on a chart and let you know what percentile they fall into. If you are that concerned, I would have a physical done so that you can get an idea of where he is and where he should be. The doctor can then let you know what course of action to take.
Assuming he exhibits a healthy appetite, unless a doctor says he's seriously underweight, he's probably fine. Busy little boys often appear thin around that age, and add a little weight as their bodies prepare for a growth spurt.
Some three-year-olds can be really husky. I have a friend whose three-year-old son's weight was on par with her five-year-old daughter's for about a year or so. Not fat, really-- he was just solid, and she was finer-boned and pretty light.
I have a nephew who is hypoglycemic and underweight and in a low percentile. However all 3 of his sisters are also in a lower percentile and none of them have any other problems. I also have 2 cousins who look like twins even though they are over 1.5 years apart. Some kids are just smaller than others. I would suggest seeing a doctor for a physical if you're concerned. But if his appetite is healthy and he's not in a low percentile I wouldn't think there is any cause to worry.