You most likely are describing an erupting wisdom tooth. The flap of skin is called an operculum, and is normal for a partially erupted tooth. The wisdom tooth most likely will not erupt fully and remain impacted. This somewhat submerged tooth will at times get infected and be very painful. You will need to have it evaluated and extracted soon.
This is likely to be temporary as the wisdom teeth can develop close to the nerve that connects to all the lower teeth. While that area is healing the nerve can be irritated and thus giving you pain signals to other teeth in the lower jaw. You should have your surgeon/dentist check for any signs of delayed healing in the sockets and or infection.