I think it does this with ringtones, and perhaps even alarms. I haven't noticed this with Pandora or other media though. I'll try to pay attention next time I play Pandora.
I have noticed that Pandora sometimes will play at a low volume when I stop listening, and then go back to it. Whichever song was playing when I stoppe doesn't play at full volume later when I start back, and I end up raising the volume. Once that song ends, it goes back to regular volume, and is now blasting due to my adjustment. I caught on to this some time ago, and now, I just skip to the next song any time this happens.
It happened on my S3, and I think I've experienced it on my Note 4, can't be 100% though.