See your chiropractor, seriously, I have a knee that does that, he works on it and it is fine. In the mean time, try wearing a knee brace to give it support.
I agree with Sassy, you should see a chiropractor. The only thing that I would like to suggest is to ask your chiropractor about using a TENS/EMS Device.
The device stimulates the muscles around the knee which can increasing muscle strength, as well as enhancing muscle endurance. Since my daughter has been using hers, her knee's have pretty much stopped slipping out.
I have the same issue. Mine stems from hyperextended muscle on the side of the knee. Rehab helps