I would just be worried that there aren't enough apps, it won't be as fun to use, and I'd miss out on apple stuff.
As an ex-iPhone user (iPhone 4), I can say that I'm totally satisfied with my Note 2. I got sick of iOS with its incremental updates and sorely outdated UI. The iPhone 5 wasn't a huge upgrade from the 4/4S IMO. The longer 4" screen was a joke and the fact that Apple was pushing this out as the best iPhone yet? Come on. I think they could've done A LOT more then just making it longer but, oh well.
The switch from iOS--->Android is a little different but nothing that you won't be too unfamiliar with. Having spent oodles of money in the Apple App store, I was a bit hesitant leaving all those apps behind however, I also have an iPad so it wasn't too bad. As far as apps go, the first you'll notice is that the Droid apps are not quite as well developed as iOS apps. Obviously there are more apps in the Apple App store then in the Android App store but that's a given. Most apps should be compatible with your phone however you'll run into some that aren't. This is where the whole fragmentation of Droid thing comes into play. Pretty much all the name brand apps like Angry Birds, banking apps, eBay, Hulu, etc are all in there. Prior to getting my Note 2 I looked in the Google App store to see what apps they had and which ones I'd be losing. You might wanna do the same just to get an idea of things. The one thing that REALLY irritates me about the Android Apps is that a lot of them are loaded with ads or will automatically place shortcuts on your homescreen for more apps. Again, really irritating since I used to be shielded from that crap in iOS. That's probably the biggest change you'll notice there.
As for the phone, I've played with the iPhone 5 a bit and, well, it's an iPhone; just the same as the previous models in terms of UI and such. Nothing really new for me which is why I went the Note 2 route. Having done so, I have 0 regrets. The screen size alone more than makes it worthwhile. I watch a lot of movies/YouTube/etc on my phone and the big ass screen REALLLLLY makes a huge difference. Even for gaming it's nice to have more real estate to use on the app. I'm to the point now where I can't even look at an iPhone without a small measure of contempt/disgust.
Taking the plunge from iOS to Droid means shifting all your media from iTunes to Kies/DoubleTwist. That will take some time TBH. It took me about 2-3 days to get everything transferred over which was a pain in the ass but now it's okay since everything has been migrated. Now it's just a matter of updating libraries once in awhile. The comfort of iTunes being your all-in-one spot for media will change. You'll probably have to use 2-3 different programs to manage all your media and such. And, with the addition of the microSD card slot, you might have to copy/paste items more than you're used to. Another thing is the UI between the two. iOS was very simplified in terms of how you navigate through the menus and operating the apps/phone functions. Droid is somewhat similar to it however the menus often go much deeper so you'll have to familiarize yourself with all the settings/menus and such.
All in all, after about a week or so, you'll get the hang on Droid. It's not as intuitive as iOS however it's more flexible so there will be that slight learning curve. Having my iPad and Note 2, I get to whore myself out to both OS's which is nice

In all honesty, migrating all your media from iTunes to Droid will be the biggest headache but it shouldn't take too long. Everything else you'll get the hang of in short order.