Wow, this post blew me away, not only because for me it did the exact opposite, but 5mg is a very small dose! I started off at 10mg and by the time I reached the right dose, I was at 50mg a day. Then again, medications work differently for people..
My suggestion would be to try a different medication. There are tons out there, it just takes some time to find what suits you best.
So the problem is that the Adderall calms you down TOO much? Is it that the Adderall is a good thing but it's too much of a good thing? What about a lower dose? Or if it is the XR, what about a stimulant that's not time-release -- in other words, one with a shorter half-life? Some people with ADHD are able to use stimulant medication only for the situations in which their symptoms are a problem, like school or work. Others need it every day. What did happen when you reduced the dose?
Maybe you could get on something else like provigil. It is also for ADHD but is not as strong and is popular among older patients diagnosed later in life. Look it up and do some research on your own about different treatment options and meds.