The only negative I could find about the Xperia Z was the camera. Other than that, it's a great phone. The Note 2 is great too but not as svelte or water resistant. Mine has survived ok so far but there's no way I'd dunk it.
Oh, the other thing to realize is that every single time you want to plug in a charger or headphones, you have to open those flaps. Every. Single. Time. For some people, that won't be much. For others, that could be quite often. Get the charging docks and you can eliminate opening that port often. Get bluetooth headphones and you eliminate opening that one too. So it's possible.