If a cache wipe doesn't solve this, it may be because you have a lot of contacts. As you enter numbers it does a lookup of your contacts to allow you to choose the, prior to entering all of the numbers, and I could see that slow data entry down.
If this is the case, the solution may be a replacement dialer from the play store. I know that Thom has one he likes - perhaps he'll reply back with his favorite.