First thing I'd do is just copy the whole thing to your PC as a backup. Then if you do delete something you find out you wanted, you can get it back.
Obviously you should leave the DCIM (camera) and any media folders that aren't empty (media, Music, Movies, etc.), as well as any folders that match the names of applications you want to keep.
You should also leave the \data directory, although you could carefully poke around and delete subdirectories in that which match applications you've deleted.
Your phone can run just fine without the SD card at all, so you don't have to worry about possibly bricking it, just possibly screwing up some applications. As long as you have a backup, even if you do that shouldn't be permanent.
