The solution was to install an app called Nova Launcher from the Play Store.
A launcher is an app which produces your phone's desktops, the dock (set of icons at the bottom) and the app drawer (you probably thought that was just part of the operating system, but no, it's all produced by an app). Since the app icons that you want badges on are either on the desktop, in the dock or in the app drawer, it is the launcher that controls the badges as well.
The desktops, dock and app drawer you currently have are produced by the launcher Samsung provides with the phone. But you don't have to use that one. Nova is one of many alternative launchers available, but it's the one I use and so am familiar with. So you install it, launch it, and then it will let you set up your desktop/dock/app drawer as you like. It will do some things differently from the Samsung launcher, and is more flexible in what it lets you do, but the basic concepts are the same. And you can switch back to the Samsung launcher and your old desktop setup will come back (you can do this any time: go into your Apps settings, select your current launcher, then where it says it is set to perform some actions by default unset that. Then when you press the home button it will ask you which launcher you want to use).
Nova's settings (which you can find by pinching in while viewing one of Nova's desktops) include settings for Notification badges. That's how you can set these things up. It has some options built into the launcher, but if you want more control of flexibility you can also install an app called TeslaUnread, from the same developer, which will integrate with Nova's badge settings (appears as an extra option in the badge styles) and can be used to customise these further (including turning them off for some apps, which I find particularly useful). But you could try just with Nova first, then add TeslaUnread if you still find it doesn't do what you want.
