Anxiety is an inherited trait. It sounds as if your son suffers from Generalized Anxiety Disorder - GAD. I suspect you are correct -denying what you see. I suspect his father is also correct - claiming his son will not have any fun in life. It's time to see the family doctor for assistance as he is unable to stop his worries. Your son will not outgrow his GAD nor will his GAD go away; but, he can learn how to manage his "fears" with proper treatment and coping tools. By the way, our child psychologist (who is involved in anxiety research) believes that the teen years are the most difficult to treat - it really is time to see your family doctor. I wish you the best ...
Have you seen "Mim's Island"? The lady in the story reminds me of your son. She has to face her fears in order to help someone else. Maybe he should visit with some children who are positive and adventurous. I am always amazed at children who have disabilities or severe illnesses (Cancer for example) and somehow they end up encouraging me.
It's probably a good idea to go to a therapist. He/she may be able to dig into any situations that may make him feel this way, and give you some suggestions to deal with it.
Good luck and God bless.