Chest pain that occurs when you exercise and stops when you stop exercising ought to be evaluated by your physician. Based on the symptoms you describe, this could be anginal in origin but it is difficult to make a diagnosis via the Internet.
Please see your doctor soon to have this worked up in more detail.
It certainly seems like the symptoms of angina with the exception of the few minutes recovery time. If you stop, it normally goes away within a minute. A heart will normally reduce its rate very quickly. I would certainly mention it to your Doctor.