This could be for a number of reasons & probably not related to your new battery, however you could try booting with the old battery.
Some of the early Desires (mine included) suffered from hardware overheating issues. The only cure is a mainboard replacement which would probably cost more than the phone is worth.
Lets look at software issues first:
I am assuming you are not rooted & that you have not changed your recovery.
First remove the battery for a few minutes. Then hold the volume down (-) while briefly pressing the power button. This will take you to the bootloader. Make a note of the pvt number displayed on the screen (it should be pvt 1, 2, 3 or 4).
Next use the volume keys to scroll down to recovery then press the power key again to select it. You should now have a black screen with an exclamation mark surrounded by a red triangle.
Hold the volume up (+) key while briefly pressing power. This should take you to the recovery screen.
Scroll down with volume keys to "Wipe cache partition" and press the trackpad this time to select.
Lastly scroll back up to "Reboot system now" and press the trackpad again. Your phone should now boot.
If the above is not successful the next option would be to repeat the above process but select "Wipe data/factory reset" and press the trackpad. However this will wipe all user data (but not the sd card).
The last software fix we could try is to run a ROM Upgrade Utility (RUU) but we would need a little more information about your phone first so let us know if the above steps work or not.