Yes, been using a firewall app since 2012 and still use a semi-retired phone at home running KitKat that's used just as a WiFi-only media player. If you just install a firewall app using its defaults and don't bother to go through the app's settings it's not going to be very effective. Also, if you're referring to the NoRoot Firewall app (Greyshirts), than yes, that's an outdated app that shouldn't be in the Play Store -- it's sadly still recommended even now in a lot of articles, a disservice to any who use it. It has not been updated since 2014 and the fact that it doesn't support IPv8 or LTE (as stated in its profile page) reflects its age. That's why I previously mentioned AFWall+, Mobiwol, and NetGuard, all apps that are still actively supported.
With root devices, a root firewall app can use iptables, the firewall service that's already integrated into the Linux kernel. So an app like AFWall+ is more of a GUI front-end to iptables. Non-root firewall apps don't have access to the iptables as it is a system-level service. They instead have to use a clever workaround and use a local VPN (local being the VPN service running solely on your device, it does not rely on any online server). So a non-root firewall app works by having all traffic go through filtering rules applied to internal VPN. The end result is more or less the same, but there is a drawback where with a non-root firewall app. If you also have a VPN app (the more common VPN that you create an account with and relies on online servers) installed that creates a conflict -- you can't have both VPN services enabled at the same time. It's one or the other, essentially you need to focus more on protecting your device (use a non-root firewall app) or protecting what you do online using your device (use a valid VPN service).
If you want to check your firewall app is working, it's not going to be easy to use any local, internal app/service to confirm/deny. Use something like Wireshark installed on a computer on the same LAN you're connecting your mobile device with. It's very highly rated as a network analyzer available for free.
https://www.wireshark.org/
https://www.howtogeek.com/104278/how-to-use-wireshark-to-capture-filter-and-inspect-packets/
Note this only applies to your LAN and having your phone connected to it. Not aware of anything similar (an external utility) to check on mobile data (cellular) connectivity.