Try soy milk or almond milk if you are not allergic to those. Cow's milk is probably causing all sorts of problems you don't realize.
I developed all kinds of problems years ago, all at one time, but the most severe was extreme gastrointestinal distress, a white tongue with a layer of junk I could scrape off, severe tiredness, night sweats, joint pain, etc. The only thing many doctors could find was very severe lactose intolerance. I started taking several lactase tablets with everything I ate and cut back on anything with milk or milk products and my symptoms almost disappeared.