I have always been told yes. A cold bottle with upset the babies belly
I keep it room temperature.
I don't heat it up.
i buy baby water and just use that at the temp it is. Some people heat it up... Some keep the bottles in the fridge. It depends what your baby likes
I use cooled boiled water reheated in the microwave for about 15-20secs and then add the recommended amount of formula.