Happens to my wife's phone as well. She works 1/2 a mile from home, but her location is all over the neighborhood (according to Google) while she's at work. The problem is a weak or non-existent GPS signal, a weak and sporadic LTE/GSM signal and WiFi all over the place from multiple ISP's.
Since her office is on the lower level of the building and has no windows, when she's there, Google location services is relying on triangulation from cell towers to pinpoint where she is. That's much less accurate than GPS and, when those signals become weak, if you have enhanced location turned on, it will try to use WiFi for location. The problem with WiFi is that is may report an access point or router or switch location, or, in the case of a VPN, you could be literally anywhere on the planet there are networks.
You could tell your phone to ONLY use GPS signals, but you may get loads of notifications about services that rely on location to not be able to locate you. Or, and I am not being a smart a$$ ... you gout get a new phone. Once the Mrs. upgraded to a Galaxy 10e, her reported location was considerably more reliable and accurate. FWIW, I still see her location jump to weird places (like in the middle of the river) when she leaves work, but my guess that when she hits daylight and the GPS signals start to take over, it gets a little confused until it establishes enough satellite signals.
