Great advice from tiredbuthappy!
Over the years, both a pediatrician and a psychologist told me (independently) that an ADHD diagnosis should not even be attempted to be made before the age of 7.
Like TBH suggested, try natural consequences and keep him really, really busy. My son was very much like yours at that age and the bathtub was used often--and he had lots of tub toys to play with including funnels, squirty things, etc. Sandbox, bubbles, sidewalk chalk, playdoh, paint (he preferred messy paint to drawing or coloring), bake with him (let him play with some of the dough), go to the stream and let him throw rocks in it, feed ducks at the lake, take him to a gym when there is "open gym" time--where he can jump on a trampoline, climb, etc. while supervised. Montessori-like activities like sorting (get a bag of mixed beans, pour them into a big bowl and then set up several little bowls and have him put the same type of bean in each of the bowls or have him sort coins or different size bells, etc.) and life skills activities (sweeping, wiping tables, washing dishes, vacuuming, etc.) are things he also loved. I found that keeping my son very busy with sensory-stimulating things was helpful.
Best of luck to you!
obviously only your pediatrician is the only one who can truly help you find the answer to your question about ADHD, but i do have a few thoughts. First, my understanding based on my experiences is that they do not diagnose toddlers as ADHD. So I suspect those people you are speaking to do not really know what they are speaking about. I have been working with young children for over a decade, many of those years in daycares and preschools, and I have never heard of a child that age who actually had a diagnosis of ADHD.
I would start by eliminating time-out. obviously it's not working. When a child is spending all day in time-out, one needs to look at what's going on. what is he doing to get in trouble? time-out isolates children from the problem and does not teach problem-solving skills or empathy. i find logical consequences to be much more effective. you spill it, you clean it. you break it, you try to fix it. you hurt someone, you try to find a way to help them feel better. And simply saying "sorry" is pretty much meaningless at this age.
to help him gain a better understand of his feelings you can try role playing with puppets. for some children it is very difficult to talk about what frustrates them. create a scenario where a puppet is angry. have your son try to help him find solutions.
you say he destroys toys because he gets frustrated with them. how are they frustrating him? are they age-appropriate? when you see him start to get frustrated do you model problem-solving for him? for example, if he is getting upset that his tower keeps falling you could say "hmm... it looks like you tower keeps falling. I wonder what we could do to help it be stronger? I wonder why it is falling. Do you have any ideas of how we could build it differently?" Guide him through the process. Don't give him the answers but rather make observations that would help him come up with the solution. For example, "it looks like that block on the bottom might be too small to hold up the rest. I wonder what we could do about that?" Guiding him like this while he plays will not only help him learn to approach problems flexibly, it will also give him confidence as he succeeds.
Are the 2 of you home all day? He sounds very bright. Make sure you engage him and provide stimulation. Go out and explore. Find some bugs and observe and draw them. Do some simple science experiements. Make oobleck or homemade playdough. children like you describe often benefit from sensory activities as they can be soothing. gak, oobleck, and playdough are great for him. fill up a dish pan with water and give him measuring spoons, cups, and funnels. take a small cup, put an 1/2 inch of water and a few drops of dish soap in it and give him a straw. watch him blow mountains of bubbles! make rainbow rice and put it in a bin for him to play with. see if you notice a difference. if you do you may learn that he needs more open-ended, creative activities. (many toys and games these days are not open-ended and tell you how to think and play. this may be what is frustrating him) you may see that sensory experiences calm and engage him. or you may just find that he was bored and needed new activities. all of these outcomes are desirable, so it's worth a try!