I believe that face pain causes CFS, or is otherwise a symptom. There are several reasons for its potential involvement, all of which suggest that your glasses, unless exceptionally heavy, are probably not causing the problem, but that you are much more sensitive to it.
In CFS the well known dysfunctions ie in mitochondria and in neural perception of pain, are going to make chronic pain more likely wherever there is stress. The head seems to be particularly involved, such as around the eyes, and the additional nasal and allergy issues also point to pain around the eyes and sinus problems. This can be cause by moulds and therefore it is possible that in a roundabout way, the sensitivity to glasses is really an eye pain that might have something to do with chronic nerve inflammation (ie a cranial nerve such as trigeminal nerve), sinus inflammation, and muscle inflammation in the eyes causing aches and pains. I think all these things interrelate.
I have fibro of about 15 years duration, and for the past approx 5 years have the same problem with my glasses. I even sprang the big bucks for the lightweight titanium ones. They are better than my steel frames, but after wearing them all day they hurt too. I would wear contacts but can't imagine sticking something in my eye! Good luck, you aren't alone.