You really don't give enough information.. are you on chemo have you had surgery how long has it been and most important do you trust your doctor?
I wouldn't worry too much if your doc isn't concerned. My Mum's docs don't attach too much weight to this count either and, at times, hers has been much higher than yours. They generally use this count to determine if chemo is effective and doing what it should.
Don't worry if you are not having chemo to bring the count down though. I think docs tend to assess your health and whether you further investigation and/or chemo etc on how you are feeling generally.