Usually it will not hurt performance, but it might affect some apps if you have a slow SD card. Use a good class 6 or better card and you should be ok.
To be sure of getting max performance install to internal memory. Not only is it faster than SD cards, but it uses ext4 which is faster than FAT32.