Do you have a community fitness center around you that might have water aerobics (or something similar)? The water helps a lot since it takes the pressure off your joints.
You might also look into workouts designed for the elderly (ie you can do them sitting in a chair). I know there's a program on PBS, but I don't know when it's on.
Eating healthy foods is another good starting point. The more sedentary you are, the fewer calories you need to survive (look on the tools & trackers page for the BMR calculator to find out approximately how many calories you need daily). If you cut out processed foods, bad fats, large amounts of sugar, and replace with whole grains, fruit, veggies, and lean protein (fish, chicken, etc) you will see a difference eventually.
I would also have a discussion with your doctor to see if there is anything that can be done to help alleviate the pain while you work on losing weight.
Best wishes on your weight loss journey! It's great that you're taking the first steps in seeking advice!