It's not so much a matter of your phone model and its age, but rather the version of Android its currently running and the apps you use. Your Note 4 is running KitKat,or Lollipop, or Marshmallow?. Since none of those versions are still supported there's no OTA security patches so as far as the operating system there are fundamental security issues involved, but at the same time a lot of that aspect is or isn't a significant issue depending on your own online habits -- i.e. if you click on links in emails you're going about things in a risky way and in this matter it's more or less irrelevant which version of Android you're running. Also there's a reality even a lot of people don't take into consideration. Even if your Note 4 was magically running the latest version, Pie, it's still a matter where your Note would be much
safer but in no way completely
safe. It's mostly fallacy that even if your device is running the most recent version of Android with all the latest security patches applied that your device is safe from being exploited and compromised. A lot of those fixes in the monthly security patches that get released are reactive measures to already existing problems that have been revealed so it's often just reactive, not proactive (fixing issues that are potential problems).
But all that is just regarding the underlying operating system. More importantly are the apps you use. Most of them are still receiving updates on a regular basis, even if Google has stopped supporting a version of the OS that doesn't necessarily mean developers stop supporting their apps. And it's the apps that are what you're using to that directly interface the their respective online services so stress that point. The apps are the direct link to your online accounts, the OS is only indirectly a factor.
So keep using your Note 4, but just be wary and judicious when you do anything online with it. Even KitKat (just as an example) with its current Android market share of just under 7% involves millions of still active users worldwide so while the percentage of single digits isn't notable as is, once you factor in it's a percentage of smartphone users it's not so insignificant.
https://developer.android.com/about/dashboards
--- and to be completely snarky and sarcastic, always be very,very cautious about having the Facebook app on your device, no matter what version of Android you're running. It's not so much a security issue but more about blatant data mining and privacy. Install a secondary web browser app, be sure to restrict that browser app's Permissions to a bare minimum, and only use that browser app to access your FB account via its web site.
