The only way to know for sure if you do have Klinefelter's is to get a chromosomal test done. My brother has KS and has grown up with many, many problems (aggression, mental retardation, hole in heart, etc). If the only symptoms that made you think that you have KS are physical symptoms, then you most likely do not have it. While on the subjest of physical characteristics, one other major physical characteristic of KS is being very tall...especially being much taller than your parents. My brother is about 6' 8" (my mom is 5'8" dad 5'10").
I am not aware of colorblindness being related to KS. but, i guess anything is possible.
I suggest you to bring up your concerns to your doctor at your next check-up. But, don't worry yourself too much. If you are able to function daily like a normal teenager, it's probably not KS.
Feel free to ask me any other questions you may have.