I hate to bump this, but, I fought this for months myself on a Note4. The absolute fix for me was:
Settings>Language and Input>Read Notifications Aloud>Calls From Favorite Contacts.
When I finally found that setting and unticked it, my volume quit dropping to ~10% everytime a favorite contact called. Only having 3 on my list, it only happened a few times a week, making things all the more confusing! I verified that was the issue, and now my volume stays exactly where I put it, period. 'Messenges from Priority Senders' *might* affect things as well, but I never tried to verify that one.
Seems the damndest of places for a setting like that which has absolute control over the volume, but thats where they stashed it!
FWIW, I'm on Marshmallow, 6.0.1 (with a constant update nag box bugging me, ughs).
Hope this cures some headaches for someone!
✌TexasGonzo
