Well the most important thing is that if you don't feel like realize you would be safe with him then it is your responsibility to keep her safe, but on the flip side you may want to just sit down and have an honest conversation with him and let him know how you feel based on your experiences so that he can see where you're coming from. If he doesn't understand or want to make an effort then at least you'll know that you have tried your best.
Is she the first grandchild? My dad was meanand gruff with us when we were kids, but I had three nieces and a nephew before my daughter was born so I got to see how he was with them. Being a grandpa turned my dad into a big teddy bear! He's still a little gruff but he's kind with the kids and just walks away if he gets frustrated rather than exploding like he did when I was going. Hopefully your dad is the same!
My dad was very loud and almost abusive towards me and my siblings but when my first nephew was born he turned into a completely different person. My daughter will be his 15th grandchild and he's just gotten more and more calm and gentle with each one.
I think a big help for my dad is that he's not around then 24/7. He gets a break where he doesn't have to be grandpa, he can just be Kent for a while. When we were growing up, he'd go to work and then come home and have kids everywhere, half the time ending up in his bed in the middle of the night.
Maybe just have him on monitor visits with you and everyone else around. But some people can surprise you and change things especially when their not around them 24/7.
If You Do Decide To Address Your Concerns Perhaps Your Husband Could Sit Down With The Two Of You. He Doesn't Have To Speak On Your Behalf Or Anything But Maybe Just Sit Next To You With His Arm On Your Shoulder So You Dnt Forget That you're Not Alone. My Dad Was Physically Abusive And Having My Fiancé Present Always Makes Me Feel Safe When I'm Around Him. Good Luck!!!
My dad wasnt the most caring and happy person to be around. Even with the other grand children and trust me theres a lot (between kids, grand kids, great grand kids and maybe even great great grand kids my daughter was I believe #61) he didnt change too much until my daughter. Out of all those kids my daughters diaper is the first one he hanged ever. That and he was always against "spoiling" the kids meaning picking them up while crying well he spoils the heck out of her. I would just see how he reacts with you around and go from there.