Loss of warranty springs to mind. And the risk to brick your phone. Usually you will be able to "unbrick" it, but the risk is non-zero.
Benefits: Use of apps requiring Froyo, App2SD (you can now move some apps from phone memory to the SD card, allowing more apps to be installed), dualtouch (pinch-zooming) with some ROMS, and several minor improvements.
Downside: Be prepared to spend a lot of time ironing out kinks, getting market app, keyboard light, keyboard layout, etc. to work. Be prepared for strange battery drain issues (like complete drain in a day on standby).
What of this will actually affect you depends a bit on luck and the exact configuration of your phone, including apps.
