Hello~It sounds like a calficied nodule to me. The fact that it has grown a little has me just a little concerned. I would see your physician to rule out anything major, it probably isn't, but it is always wise to err on the side of caution.
Although It is nothing to worry about If it just disappears or bursts naturally with time on its own, and all the puss etc oozes out. But it may become an issue when there are signs of pain or itching, which may lead to an infection or may require treatment by a doctor who may slice it open. So, it is better to start applying a specific ointment may be an anti fungal or anti bacterial of some sort, as recommended by a doctor to avoid any future possible complications or at least help it recover before the complications may arise. Some people would also heat some dry cotton and pat on the lump to allow the puss ooze out. Or else also hot saline water and cotton soaked in it, and pat on the same. Anyways, make sure you maintain good hygiene, avoid deodorants on the area and have regular baths. Also avoid shaving the arm pits for a while, trimming is better, to avoid any infection through cuts. Sometimes it may be just an infection in the hair follicle in the skin or some new hair starting to grow and sometimes it may be serious in exceptional cases. All the best!