It isn't "running" unless you activate it and start using it for something. It will just take up space in your RAM until something else needs the space.
If you root your phone, you can use a program called Titanium backup to "freeze" unwanted apps, which often can be more stable for the phone than removing the bloatware. Why, who knows, but it also gives you the ability to return the phone to "stock" state if needed, without a lot of hassle.