The 4G icon stays lit on Wifi (but not showing data transfer) to enable Voice Over LTE - VZW's first step towards shutting off their old CDMA/3G towers. To enable VoLTE, you need to login to your VZW account and add the "HD Calling" feature. Follow the Q&A on the VZW site for more details.
There is no extra charge for making that switch and 'HD' calls are part of your normal voice allowance. The Q&A has info on how it deals with switching between 3G and VoLTE. They do tell you to be prepared for dropped calls if you move out of 4G range while on a HD call, but VoLTE can be disabled from your phone if you are in an area with weak 4G to force it to stay on CDMA. (In a pure CDMA area it will automatically use it, so no need to turn off VoLTE just for that).
BTW, the VoLTE option will show up under 'More networks'->'Mobile Networks' after you select the feature, power down the phone fully and power it back on. Be patient after the restart since it can take 10 - 20 minutes or more to show up.
That feature also turns on video calling - no 3rd party app needed. The video part of the call uses your data plan while the voice counts towards your minutes, but the good thing is it will seamlessly switch the video over to Wifi when you connect to a Wifi hotspot and save on your data usage.
So, the days of permanently grayed out 4G icons are gone. Just watch for apps that have a problem selecting the right interface and try to use 4G even when on Wifi, since both interfaces appear to be active. You should never see the arrows under the 4G icon light up when on Wifi unless you have one of those confused apps. VZW Visual VM is one of those that appear to always use 4G, but I'm not sure if it's intentional or will change in future. (Incidentally, enabling HD Calling forces you to select Visual VM Basic!)
So far 'HDHomerun Viewer' is the only app I've found that's broken by the 4G interface staying on - workaround is to turn off mobile data to force it to look for tuners on the Wifi network. You're going to have to keep reporting these odd-balls to the vendors as you find them so they can fix it.