Exactly. At 30 weeks, i was so little. I just looked like i had one too many cheeseburgers. I gave birth at 38+3 and my daughter was 6 pounds, 13 oz. It's nothing to worry about.
If your doctor isn't concerned, you shouldn't be either. You can always bring it up at your appointment. But a small bump doesn't equal a small baby and vice versa. Some women just carry differently.