Don't the navigation options on the Fire remote work for it? Although ES like some other apps were really intended to be used with a touch-screen, like a phone or tablet. If ES doesn't there's probably not a lot you you can do, apart from finding another file explorer that does, or you use a Bluetooth mouse or keyboard with the Fire, or the ES developers in Beijing make their app more Fire TV compatible.
ES is not the best app anyway, like it's phoning home your personal stuff to China, i.e. it's spyware.