Get to a urologist and have him take a look at it. It may have had some damage or scarring that is now causing issues.
Another possibility is an inguinal hernia, from either an inherited condition or from heavy lifting. Part of the intestine goes down the inguinal canal. One testicle may have pain, it's all related nerves down there.
Ignore the first answer regarding testicle size, many men have one larger than the other, one hangs lower, etc.
Hi,
It may be varicocele of the left testicle. Is the size of the left one similar to the right, or much smaller? In any case, you should consult a urologist for detailed diagnosis.
Solace