My phone doesn't have this ability either, but I use Go Sms Pro, then I disable incoming notifications for the stock messenger in its settings, and go into Go, choose notification settings, then default notification settings, scroll to reminders and check the box to enable reminders. Then you can even customize the sound and time interval that you get a reminder for unread messages.
Edit: For voicemails, I am not sure. Maybe someone else knows.