Ok, have to admit I had to chuckle at this thinking you're lucky that is the only *cat* thing they are eating. LOL!!!!
Seriously, it won't hurt them now and then as long as that isn't their regular diet. At first, I thought you meant that was their regular food and, yes, it may give them the runs.
By the way, one side effect has been known, from dogs eating cat food in any quantity, that is diarrhea. It's basically too rich for them.
Oh a tiny tiny bit of catfood isn't going to hurt him. It isn't exactly toxic for dogs, it is just it often contains taurine which dogs can't really handle in any quantity. A "few kernels of kibble here and there" aren't going to do much harm unless there is any allergy to what the food contains or something.
But I would certainly not feed cat food regularly, or in place of dog food. Cats and dogs have different dietary requirements. They are different species.
Dogs are pretty omniverous and can get away with a tiny bit, but no more than that. I would make sure he can't get to the cat's food dish constantly. (poor kitty, anyway! LOL!)