I developed all kinds of problems years ago, all at one time, but the most severe was extreme gastrointestinal distress, a white tongue with a layer of junk I could scrape off, severe tiredness, night sweats, joint pain, etc. The only thing many doctors could find was very severe lactose intolerance. I started taking several lactase tablets with everything I ate and cut back on anything with milk or milk products and my symptoms almost disappeared.
Try soy milk or almond milk if you are not allergic to those. Cow's milk is probably causing all sorts of problems you don't realize.