Yes, having a defibrillator can cause extreme anxiety in some people and panic attacks are quite common. There is a good online support group for people with defibrillators www.icdsupportgroup.org. You will find that what you are experiencing is completely normal and does get better with time if you talk about it and try to work through the anxiety.
There are several anti-anxiety meds out there and depending on your specific heart condition these might help too. Talk to your doctor.
Check out the ICD support group. Knowledge is power!
Take care
I don't think the little unit itself can cause panic attacks since it sits quietly until it's needed. But I have read of people that get very anxious about having a foreign object placed in their body. Like any fear or concern, talking it through with someone (your doctor or a counselor) may help put your mind at ease.