I would probably seek an OB or RE opinion sooner then later. Of course it's very possible, but being 43 does lessen our fertility. Combined with a low sperm count, its a bit tougher. Your Dr.s can help you with options and things you can do to increase your chances. Good luck.
I think its possible for you to have a baby but, considering your husband's low sperm count - you might need some help like ivf , or iui. I have a friend who had low sperm count and his wife has endometriosis. They went through the ivf route and now have twin boys. Good luck to you - hope we hear some happy news from you soon ! SSBD to you.