You must contact Verizon and request cancellation of Visual Voice Mail or they will continue to charge you the monthly fee whether you use it or not.
BTW: How do you like the GV integration with the Droid? I am considering using my GV invite to get started but have been holding off until I know what functionality it changes / how it improves phone experience.