These symptoms of feeling to urinate associated with kidney stones can be due to strangury. It is painful, frequent urination of small volumes that are expelled slowly only by straining and despite a severe sense of urgency. Get the size of kidney stones evaluated. Drink plenty of water and talk to your nephrologist regarding the treatment of kidney stones which include pain killers, trying avoiding taking calcium rich foods. If the symptoms persist then surgical intervention is needed. This includes, ureteroscopy, ureteral stent placement, ESWL and percutaneous nephrostomy.
I suggest you to get an evaluation done from a urologist and discuss the various treatment options. Till then you can take oral anti-inflammatory pain killers like Ibuprofen.
Hope it helps. Take care and regards.