No court would take a baby from its loving biological mother and peaceful home to share custody with relatives that are not even the father. (The biological father would have more rights, but not for example his mother and father. Grandparents have surprisingly few legal rights in the U.S. although this is changing.)
This is more of a social question with long-term ramifications for your child and his family relations than it is a legal question. Just because your ex-boyfriend (may he find peace in heaven, as he surely did not on earth) was awful, would it automatically hold that his mother or father were also awful? Would it benefit your child to have this group as an extended family if the DNA test shows that your unpleasant ex was the father? And how would it make your boyfriend feel, who is the true father to this child no matter whose sperm made the baby?
If you want to tell me the dates of the sex and other details I can help you try to work out what the chances are that your ex was the sperm donor. It might help you decide what you want to do about this family's request. If you have them, please mention:
-first day of last period before getting pregnant
-dates of sex with guy #1
-dates of sex with guy #2
-date when you learned you were pregnant and how you learned it (home test?)
-date of first ultrasound and whether they gave you an estimated due date at that time (and if so, what it was)
The estimated due date from your first ultrasound would be the most helpful piece of information, along with the date(s) of the sex with your ex and with your boyfriend around the time when you think you would have gotten pregnant.
Regarding getting advice about whether to go ahead with a DNA test, one way to handle this if you don't like your ex's family is to let them be the ones who initiate any legal action to get this to happen. They might be told by the court that they don't have what is called "standing" to even ask you, and then it would be a dead issue.
But I would also talk to a counselor, especially if they are an OK family who just happened to have an abusive person for a son. You should think about what is fair here, and the feelings of your ex's mother if she is grieving. I am not of the opinion that the characteristic of being abusive just "happens," so I would be leery of what the family's dynamics might be. But if you do think a baby from your ex would matter to his family a lot, talk this over with a clergyman or a social worker or counselor for advice. You would need to tread carefully, possibly working out visitation with you there only, if you decide to go ahead, because as I said, the habit of abusiveness can come from how someone is raised.
I think you hold all the cards. If you want a legal advisor, there are legal clinics associated with all the major law schools who would be glad to help you with this for no fee or a really small fee. If you want an emotional advisor, try a church if you go to one, or a counselor. Write back if you want help figuring out if your ex could really be the dad.