Rashes like eczema, fungus, or scabies are bilateral and more or less symmetrical (affecting both sides.) Along with many other considerations the fact that yours is on one side for so long rules these out.
Your condition does not sound like anything contagious. (After 4 years, how could it be?)
The soreness and loss of hair are no doubt secondary to scratching. Stop scratching--and all the symptoms will go away.
But how can you stop scratching?
First, by accepting the fact that you don't have a serious or contagious disease. Second, by recognizing that when you start worrying about a certain part of the body, that part starts to tingle and itch, which makes you scratch, which then starts a visious cycle. Third, by applying no active medications to the area (cortisones, antifungals, etc.) Instead, moisturize daily with Aquaphor ointment (over-the-counter). Also, get some Sarna lotion (also over-the-counter) and apply that as needed for itch--as many times a day as you need to.
After you've done this for one month, show the condition to a dermatologist--one of the same ones, if you have no other choice. This will help confirm that you don't have something else going on. Plus, by then your skin should look and feel better, and you'll see new hairs growing in. You may also want to consider psychological help in dealing with your fears of idsease in this area. But nothing succeeds like success, and once you see your skin getting better, as I'm confident you will, then that will bootstrap things so you can feel good and get on with your life.
Best.
Dr. Rockoff