Not sure what you mean by a voicemail section in the dialer, but I can confirm that I don't see anything in there that specifically deals with voicemail.
On the other hand, I can also confirm that voicemail notifications definitely work on my N5X: I received 3 different voicemails the other day. I thought the notification wasn't clearing properly when I listened to a voicemail then saw the indicator again next time I looked at the phone but it turned out to be a new voicemail.
All a bit of shocker as it happens: I never had a voicemail in the 3 years I had my N4 - I thought(/hoped) my plan didn't support them or that they had somehow been turned off.