You may be describing hymenal tags, also known as hymenal caruncles, or p***ytongues informally. One of the best things you can do is compare.
It sounds, from your description, that it COULD just be hymenal tags, also known as hymenal caruncles or, informally, p***ytongues. If you don't mind looking at a NSFW site, try going to a tumblr like p***ytongues.tumblr.com or hymenaltagslover.tumblr.com to see the variety of hymenal remnants and see if any of it looks familiar. If so, you might be relieved —and also get some education on these sites about hymens and how they look in various women in various stages of life and sexual activity.
What you thought was "weird" or "embarrassing" might be beautiful and sexy!
A lot of vulvas look like a hamburger with fat lettuce hanging out. A lot of women get their idea of what their lady parts should look like from either childhood (when the vaginal lips are still inside the vulva) or from porn (which is into genitals on women that look childish). Don't buy into the notion that anything about your genitals is odd or unusual. There are lots of ways for genitals to look, and except where they are mutilated by surgery, they are ALL normal. As men are often told (but don't believe), it's not what you look like but what you do with it that counts.