I'd guess it was the underwear itself, not the person, that he was reacting to, were they particularly silky or had been washed with something that smelled good? Kids at a young age can seem a bit fetishistic to adults (who are of course judging by standards that shouldn't be applied to kids) in that they can like to touch their mother's armpits or feet or want lots of skin-to-skin contact, kids also historically love those blanket edges that are really satiny and will often soothe themselves to sleep at night stroking that part of the blanket. If the grandma's underwear were satiny I wouldn't find this a surprising thing for a kid to like.
In your position, I'd just sort of keep a weather eye on the question later as he begins to get his hormones. It isn't much to worry about even if it develops into more fetishistic stuff (he's going to be who he's going to be) but it might help you if things like this don't catch you by surprise when he's older. In other words, be ready to be accepting and calm no matter what happens in his sex life later, but the need to handle something surprising probably won't even occur. :)