Children who display such anxiety become fearful at the prospect of being away from their parent or familiar surroundings. This can be hard on parents, because it can be stressful with a child seemingly 'glued' to us. But it is important to maintain our equanimity and keep the perspective of empathy for the child who is so fearful. Many, though not all, children who display ADHD also display other conditions, such as anxiety. If you son is treated by a psychiatrist (vs by his pediatrician), be sure to discuss this with him/her. If your son's ADHD is managed by his pediatrician, arrange an appointment with a psychiatris to discuss the anxiety. Anxiety disorders can be treated, sometimes by therapy alone and sometimes by a combination of therapy (usually cognitive-behavioral therapy) and medication.
Maybe he is afraid to be left with whoever you are leaving him with?? Why don't you take him with you?