Have you given Homoeopathic medicine a try? It might sound like total B/S, and don't ask me HOW it works! but it can really help to relieve symptoms, has no side effects, and is safe. Very often it helps tremendously.
I haven't tried it, but found chondroitin and glucosamine and/or MSM to be very helpful if you're looking at non-prescription remedies. You won't see an immediate difference and I understand these products are more for prevention than curative.
If your dog has severe arthritis, you really need to get a prescription anti-inflammatory like rimadyl or deramaxx. I had two dogs who took those medications for years with absolutely no trouble. The deramaxx is more expensive, but it also worked when the rimadyl was no longer effective.
Unless your dog is extremely crippled, mild exercise like short walks or swimming can help keep him going longer with fewer symptoms. Strong muscles can take some pressure off the bones and joints.