Please state what power adapter are you using. It's the charger that provides the power to recharge the battery, if you're using an older 5 watt adapter (5 volt, 1 amp) that will be a limiting factor. Most newer power adapters are now at least 10 watt (5 volt, 2 amp).
Just to get a sense of scale, an E4 has a 2800 mAh battery (or 2.8 amp/hour) so, to split hairs, even a 2 amp charger will be putting out a slightly less than optimal amount of power to be in parity with battery's capacity, but practically is adequate for most situations.
But if you do have an at least 10w power adapter, has this problem been happening for a while now or did it just start happening recently? If recently, have you restarted your phone? A reboot often solves a lot of one-off glitchy problems.
A plus point with an E4 is has a user-replaceable battery if that becomes a last resort option.
