In particular for android devices:
Here some information on the operation related to android
Essentially, Java is usually translated from source to Java bytecode (an intermediate step between Java itself & processor specific machine code) when the app is compiled. When it is executed, this bytecode has to be interpreted into machine code. What JIT does is cache the results of such interpretation, so that, for example, if a particular section of bytecode is executed multiple times, it will only have to be interpreted that first time. JIT for Java has been around for a good while.
Android uses their own special Java Virtual Machine, which we hear of often - the Dalvik Virtual Machine. This works on similar principles to a typical JVM, but it produces a type of intermediate bytecode that is optimised for execution on devices with limited resources. This code, incidentally, is called a Dalvik Executable, which explains all the .dex sufficies in Android.
As this Dalvik Virtual Machine is relatively new, the further refinement of just in time compilation for it is only now reaching a stage where it's production ready - or at least hopefully will be soon. All pretty exciting as it can boost performance by around 200-300%