I'm facing the same issue with my supplied 4GB card. I had raised it with HTC. I think they are aware of the issue, but they don't have a solution as of now. The guy had initially suggested deactivating the backup application, which I had already done. Here is the exact second reply they gave me:
Hello again Kaushal We've seen this issue reported for a couple of Desires and if these steps didn't solve your issue I was wondering if you'd be willing to have your device sent in to our repair centre for some testing, this way we could get to the bottom of this issue. Let me know if you'd be interested in this and also in that case what your phone number is and I'd arrange for someone to contact you further. Best regards, Dennis M HTC
I am not to sure about sending my Desire away for this testing as I don't know how much time it would take. So, I haven't yet gone ahead with this. Plus, as we know it is not easy to reproduce this issue. Sometimes it happens twice a day. And at times it doesn't happen for 2-3 days.