While typing this, my battery temperature is a little lower, but it is early morning, the ambient temperature is quite low and the phone has been out of my pocket for a while (without a case). So, I would not worry too much, especially if you are browsing websites with flash or other animation.
Regards,
Eric Ritchie.