If you plug a 5 Volt charger to a 3,75 volt battery, aside from anything else, you're going to destroy the battery. (The voltage appled to the battery, and the current it's allowed to draw, depend on its state of charge, moment by moment.) There's circuitry between the charger and the battery. Buy an external charger that can charge the battery o8utside the phone. Look on Amazon for cellphone battery chargers.
Aside from determining which terminal is positive, you have to determine which terminal is negative (there are 4 terminals). The others are sensing terminals (to determine the state of charge), and if you apply 5 volts to them you could (depending on what polarity you apply to what) render the battery permanently unusable. If you're lucky. Ask Sony about how hot a Lithium Ion battery gets when the lithium is actually burning. (Buying a new phone is cheaper than a prosthetic hand, unless it's Sony's fault and they're paying your medical bills.)
This is one of those "don't attempt this at home" things. It's even a "don't attempt this in a cellphone repair shop" things. (If I had ever caught one of my techs trying to do what you're planning on doing, the only question would be whether to fire him [preferably out of a cannon], shoot him or call the police. Badly mistreated LiIon batteries make small, but effective, bombs.)
