Hi there,
Let me first say that your son's fears about ghosts and his fears of the dark are very common for young children. I am not an expert in child development, so I cannot articulate what may be going on in his mind, but I can assure you that it is very common, and likely to be a phase that he grows out of. It is partly a product of what developmental psychologists call "the separation-individuation" phase, where children begin to explore the environment on their own for the first time. For a little kid, as much as he or she is eager to do so, he or she may also be terrified. One could see getting into bed with you and your husband as a way of calming his fears about the scary world. The other part may have to do with the emergence of "scary" feelings inside of him that he is having a hard time dealing with for the first time. It is very common for young children to "project" or attribute to the outside world feelings that they are having but are too hard to deal with (adults do this too, all the time). So it is also possible that your son is thinking ghosts are wanting to attack him because he is feeling these feelings of aggression/anger towards others (this is a normal part of development, I can assure you). Getting into bed, then, allows him to be reassured that his own aggression/anger is not destructive (in other words, you may be the object of his anger and his fear that his anger will destroy you). This all may sound a bit absurd, but keep in mind, we're dealing with the mind of a young child, and rational thinking and logic has not developed yet.
Just a practical suggestion. I don't know how big your bedroom is but is there room for a child's bed in there? While your son is getting through this phase, a halfway point for him instead of sleeping in your bed would maybe for him to sleep in his own bed but in your room for a while, so he knows that mum and dad are there to protect him all night.
This is so common in children of this age as to almost be the norm. What is and isn't real and when it is right to be scared are still all being worked out, and with sleepiness we all enter a world where reality is a bit warped. For a young child this can be terrifying. I am thinking back to my own childhood, where I would lie awake, staring in horror at the innocent furniture in my room, thinking a ghost was hiding in the wardrobe or in a dark corner. It is a stage of development, and mum and dad are all that is safe in a scary night time world. Being in your room for a while may help. He is not making excuses to be with you I think, the fear is very real, and at this age you do not need to worry. But personally I always get nervous about the logistics of young children sleeping in a bed with their parents, although lots do without problems. I just worry about them getting into breathing difficulties if one adult turns over on them. But I did not intend to throw another anxiety into the mix! Most children at some point sleep in their parents bed with absolutely no problems.
Really, at this stage just accept this for what it is, and be there for your son when he is frightened. If in another couple of years he is still having issues, then you can start to feel concerned, but not at this age. Let him be with you, and understand that with all the reassurance in the world the fear is probably still going to be there, as long as he knows you are there to protect him that is enough.