No, don't worry about it.
This isn't Windows 98. Android is Linux-based, and Linux memory management works on the principle that "unused RAM is wasted RAM". Hence the system will always cache recently and frequently used apps in RAM, which means that it uses less time and power to load those when you need to use them. If it needs the space to load a new app, it will clear some of the hibernating apps out of RAM to free space, but otherwise they are doing no harm and will improve the efficiency of the phone.
You should avoid "task killer" or "RAM optimiser" apps, because despite their claims what they actually do is slow the phone down and reduce battery life. My previous phone had a similar amount of RAM, and usually ran with less free than yours has. It isn't a problem.
