Really short Luteal Phase

I am a 30 year old female who has been TTC for about 8 months. I have been off the Nuvaring for about 9 months. I do chart my cycle. Ever since coming off the nuvaring, I have been having a very short luteal phase. I usually ovulate around day 19 or 20 and then about 8 days later, my period starts. In the time between my ovulating and my period, I usually spot. I have had no other medical issues. I voiced my concerns about how quickly my period comes after I ovulate and my doctor just brushed it off. I realize that there is an adjustment period for the body after coming off hormonal birth control but I thought by now my cycle would be a little more normal.

Is this something that can be preventing me from getting pregnant? Should I go to another doctor?