It's a bit strange but I'm just not feeling the 5x. It doesn't seem like much of an upgrade. I've never had a fingerprint scanner before, doesn't interest me. Reviewers seem to have issues with the processor, users seem to have issues with lag and stuttering. And its bigger.
It's probably because the nexus 5 is perfect for me.
And I think that's probably the answer right there. Get another nexus 5! The only drawback seems to be no further updates, but it will still be way ahead of most android phones for a while yet.
The lag and shudder issues were taken care of with an update earlier this year. The complaints about the processor are largely compared to the Nexus 6P; since you find the 5 perfect you shouldn't have an issue. Again, I think you'll be surprised how much the 5X reminds you, in daily use, of the 5 -- which is to some degree was a complaint (seemed to similar to a 2 year old phone some users were replacing). It is a bit bigger, and a bit thinner and a "whopping" 6 grams heavier (about a quarter of an ounce). Most people who switched from the 5 to the 5X don't notice the difference in size, the two phones feel very similar.
As for processor, the 5X does have a better processor than the 5. The "issue" is that the 5X encrypts all data, by default (though it can be turned off). So, for a time, there appeared to be some type of error in the decryption process that caused the phone to lag at times. They fixed it with an update, as I mentioned, and the phone works well.
One other difference I forgot to mention previously, the 5X has all current LTE bands used by US carriers -- such as Band 12 -- the 5 has about half the LTE bands.
As for the fingerprint sensor, I can't guarantee you will find it a big deal but most people do. It is nice having a "lock" on your phone, yet still be able to unlock it in little more time than it takes to pick the phone up.
Having said that, as you point out, there are some differences. What you might consider, order the phone, try it out and see how you like it compared to your old Nexus 5. My personal guess is you'll be quite happy with it -- extremely similar but with new/improved features and Android updates. If you don't like it after a week, send it back under the return policy and go buy yourself a used/refurbished Nexus 5.