Sure you could live without Google-server based apps on your phone but I don't think it would speed up your phone AT ALL. I also don't think that there's any way to delete your Gmail account from your phone (though you could completely remove any data from it and otherwise abandon it, leaving it as an empty shell).
Personally, I'd hate to live without:
* Google Play Store (though Amazon apps store might suffice), and
* Google Maps/Navigation, and especially Waze (which is owned by Google).
* (I'd slightly miss YouTube.)
Otherwise, you could easily live without Google's services. For example,
* I uninstalled Google's Music app replaced it with MediaMonkey.
* I have no interest in Google+, Hangouts, etc. so they don't exist on my phone.
* You can replace Google Search with Yahoo, Bing, et al.
* You can use Hotmail, Yahoo Mail, et al, with their apps or a stock email app instead of Gmail.
* You can use Hotmail to sync your contacts with your PC, Outlook, and keep them away from Google. (I do.)
* You can use OneDrive, Box, et al, instead of Google Drive.
* You can install an alternative browser, like Dolphin.
You'd still have a fully functional phone that would arguably be superior to iPhones, Blackberries, et al.