Someone can correct me if I'm wrong, but in a nutshell I believe fragmentation as it relates to Android is how many different versions of Android are running on various phones across carriers.
In other words, there can be 6 phones released in the same month and 2 of them might be running Android version 2.3.2, 2 of them might be running 2.3.6 and 2 of them might be running 4.0.2.
So I think it just refers to all of us not running the same OS even though the phones were all released at the same time.
Apple for example doesn't really have fragmentation because for the most part all iPhone users are running the same version of the OS. And if they aren't they have access to the most updated version.
Once again, someone can correct me if I'm wrong on this.