Back in 2010 or 2011 several of us tested this: ran our phones with GPS on and off and compared lifetimes. It makes no difference at all.
As Brian says, GPS is only used when an app that requires it checks location. So sure, if you do something like set up a route in Google maps, turn navigation on and then leave the phone like that it will drain quickly (and not just from the GPS). But unless you do something like that it has no impact.
(I actually go one further and use just GPS, no network-based location. And I deny location access to any app that I don't think will need it, and allow most others to only access location when I am using the app. But that's for reasons of privacy rather than power drain).