If you have wifi switched off, try switching it on. Is this better? Provided it can see a wifi hotspot it can get a more accurate location from this, even if it doesn't connect to it. Wifi positioning is actually pretty good and at work has me pretty much sat at my desk whereas with cell positioning only it has me about 500 metres away which is useless for things like public transport notifications from google now.
If you have gps on, not all apps will use it (I don't think google now does if checking location in the background) plus it only works outdoors mostly so wifi may still be your best bet.
If you have gps on and you're stood in the middle of a field and your position is still rubbish then I'm not sure. If it doesn't imporve after a few minutes then I'd consider a factory reset