My boobs are usually pretty tender for a couple of weeks but it always takes months for my milk to dry up. I don't know if that's just me...
I assume this means your milk has come in and there getting engorged! Should hurt long, keep a bra on 24hrs a day even in bed and the shower it really helps to 'dry up' your milk supply!!