They can say what they want. I would still go ahead and set limits. If they complain about you not being a gracious host the week after having a baby, it will be pretty obvious who is the more reasonable party...
They will definitely get offended lol my aunts and uncle's wives will start bitching believe me.
I would suggest telling them now that you would like to have some rest/privacy after the birth and to lay out some basic ground rules (e.g., no more than a certain length visit, no visits after x pm). That, and you don't *have* to answer the door. :)