I cannot believe a vet would tell you something like that. It doesn't even make any sense. Sounds like a more serious problem and my best advice to you is to take him to another vet as soon as possible. The way you describe this, it sounds to me like he is very dehydrated and this is being caused by a far more serious problem. Your kitty needs some blood work done. Your kitty could have a great number of things. Has he been vaccinated? That pale color that you see could be from either dehydration or he could be Jaundice, meaning his bilirubin values are high, indicating damage to the liver. Please make an appointment with a vet who knows what they are doing, and in the mean time, try to keep him hydrated via a syringe. Also, you could try baby food see if that stimulates him a bit. If not you will have to syringe feed him. He is probably very nauseous and most likely will not want to eat at all, so you'll have to force him to eat with a syringe until a good veterinarian can see him.
Poor kitten. I hope he will be ok. Keep us posted. Wishing you the best of luck.
I have to concur with PrettyKitty and I urge you to get your little guy to a veterinarian today if at all possible. Cats are not designed to go for more than 24 hours without eating. Since your guy is refusing food, that's enough of a reason to go in for a good physical, bloodwork and maybe even xrays.
Good luck...let us know.