What kind of hosting is this? Shared hosting or something like google app engine, or Amazon EC2? Shared hosting is pretty useless to developers, but if it is the latter, I can suggest python 2.7, PHP 5+ fastcgi, etc... Also, make sure you have both a user-friendly interface, as well as a way to tunnel in via SSH and FTP.
Also, mysql databases are a must. and I personally prefer postgresql database support also.