I agree with Nicoal, and would like to add that if when your cat was neutered, any testicular tissue was left behind (it only takes six cells of testicular tissue to trigger a mating response), he will continue to try to mate. In a case like that, he cannot sire kittens, but will still try. Do as Nicoal says, then ask your vet about my comment... it's possible that this tissue can be removed if left behind. (and doing so should be free if you go back to the same vet) - Blu
How long ago was your cat neutered? It can take some time for them to calm down, and some are so high enery they just don't. As far as letting him outside, you should wait a few weeks because the sperm can stay in their system for a little while, so even though he was neutered, he can still impregnate females. Cats are very territorial, so they could still attack him whether he is neutered or not. The best course of action is to keep him inside. It's much safer for everyone involved.