One thing is clear: even though we call them cell phones, the phone feature is becoming a smaller fraction of what we use these devices for. It once made sense to shrink a phone if all it could do was make calls and store contacts. The features that make the device smart are more pleasant to use on a bigger screen, so naturally, our phones are increasing in size.
The trend toward larger screens is consumer-driven. Once a device becomes too big, it will be clear to the manufacturer that they've crossed the preference threshold, and we might see size stabilize. Maybe we're almost there; maybe not.
People have been trying to call the "perfect size" for quite some time. It used to be in the 4" range. Clearly it isn't. Just wait and see. And don't worry. With the exception of Apple, manufacturers realize that one size does not fit all. We won't all have to use Note-sized phones if we think that's too big (or small!).