Yes, you may have an infection. Go back to your dentist and get it checked out. You may need a course of antibiotics and they may need to go in there to clean the area up.
Whoops! Brown usually represents dried blood. You may also have an infection which will need treating.
You may have a dry socket.
I would advise following up with your dentist.
If it is a dry socket no amount of pain relief will help the situation.