Android More Stable than iOS? (Phandroid article)

[s.m.knipe]

Now, without starting a flame war (stick to facts/issues please, this isn't the Republican primaries after all ), how interesting/believable is this?

Android vs iOS – Which Is Really More Stable? New Data May Surprise You

I've said this for a while but there had been no good methodology for testing (as Apple apps do not notify of a crash), nor a quantified study of it, but it looks like someone took it on. What does anyone know of this "Crittercism?" Discuss...

 

[trophynuts]

Would you mind if i moved this to the Android Lounge?

Android Lounge - Android Forums

 

[s.m.knipe]

Would you mind if i moved this to the Android Lounge?

Android Lounge - Android Forums

Oh sure, sorry I should have thought about that

 

[EarlyMon]

Actually, I think the Lounge is better suited, as we've usually kept iOS/Android comparisons there.

The results seem very reasonable. Looking at the original Forbes source for the Phandroid article, we see that the data are normalized, and both cite what would seem to be a valid metric to ensure proper sample size - over 214 million app launches were monitored.

I posted this as a lark in another thread and was surprised to find that not everyone knows that iDevices can lock up with a bad app, and I suspect that many people have never considered this as a difference in the two competing operating systems.

 

[s.m.knipe]

Actually, I think the Lounge is better suited, as we've usually kept iOS/Android comparisons there.

The results seem very reasonable. Looking at the original Forbes source for the Phandroid article, we see that the data are normalized, and both cite what would seem to be a valid metric to ensure proper sample size - over 214 million app launches were monitored.

I posted this as a lark in another thread and was surprised to find that not everyone knows that iDevices can lock up with a bad app, and I suspect that many people have never considered this as a difference in the two competing operating systems.

Yeah I should have thought about the lounge for posting it, thanks guys for moving it.

I didn't know they would lock up either, that's interesting given the rather angelic status iDevices enjoy in the blogosphere.

 

[EarlyMon]

Indeed.

For those with apps for the forum, or just in a hurry, here's the tl/dr snapshot -

Because the pie has so many slices, it's a little hard to see the arrows, but basically, it's showing well over of 75% of crashes monitored going to iDevices.

The other thing that's interesting is that Apple propaganda to the contrary, people are not marching along in lockstep with their iOS updates and therefore, iOS is every bit as fragmented as Android.

So, that's two myths killed for the price of one.

 

[s.m.knipe]

Indeed.
Because the pie has so many slices, it's a little hard to see the arrows, but basically, it's showing well over of 75% of crashes monitored going to iDevices.

The other thing that's interesting is that Apple propaganda to the contrary, people are not marching along in lockstep with their iOS updates and therefore, iOS is every bit as fragmented as Android.

So, that's two myths killed for the price of one.

I find it a little odd there isn't much commentary on this (given some of the other volatile comments in less factual threads) ... as Earlymon points out above, the findings/implications of this report are huge to the myths surrounding iOS and Android.

For uprightness it should be noted that in that pie chart above "Android other" includes 16 previous iterations, bringing the total number of Android versions tested to 33, to the iOS number of versions at 23. Still, just looking at the percentage those 16 other versions collectively represent, it is easy to see why they were collapsed. Plus, that does nothing to undermine the fact that iOS still is very fragmented.

 

[EarlyMon]

You know, the more I think about this, the more I think we were all right - I'm going to leave this here for discussion, pin it to the Android Lounge sticky, and extend the link from the Nexus forum so that it doesn't evaporate too quickly.

Aside from the information-for-its-own-sake side, this does deserve a marker for those comparing Apple vs. Android flagship phones, and many visit the Android Lounge looking to know the pros and cons of changing away from iOS.

 

[jerofld]

So if that chart is to be held as a source of comparison between iOS and Android, iOS 5, the latest iOS, has had more crashes than all of Android combined. And I believe Android still has more market share than iOS?

Wow...simply wow.

 

[gallandof]

now maybe my customers will believe me when I tell them I had more freezes and crashes with my iphone than android. (I have gotten written formal complaints that I put down the iphone, and focused on it's faults over android). If only I was allowed to use studies like these during selling

 

[s.m.knipe]

If only I was allowed to use studies like these during selling

Why can't you? Corporate policy? When I worked at Sears we couldn't throw "Consumer Reports'" name around, BUT we brought in the magazines and had them open to the product pages so the customer could read it themselves. Maybe print it out?

PS people actually got sufficiently upset to complain about that?!?

 

[gallandof]

Why can't you? Corporate policy? When I worked at Sears we couldn't through "Consumer Reports'" name around, BUT we brought in the magazines and had them open to the product pages so the customer could read it themselves. Maybe print it out?

pretty much, we aren't allowed to use any sales tools or outside marketing that the company hasn't pre approved. they changed it around because a lot of employees were bringing in android or apple fan sites as fact, wasn't enough un-biased info

Edit: yup. one women was actually yelling because verizon hires people that don't like the iphone, and that everyone should like the top selling device, otherwise people are going to be sold phones that they didn't come in for. she was upset because she bought a nexus, but her husband wanted her to get an iphone so he could play with it.

 

[s.m.knipe]

pretty much, we aren't allowed to use any sales tools or outside marketing that the company hasn't pre approved. they changed it around because a lot of employees were bringing in android or apple fan sites as fact, wasn't enough un-biased info

ah, true. I forgot about all the mis-information that flies around fan sites. Still though, I would think this is fairly credible and would be able to be approved. There has to be a bigger profit margin (aka more reason to sell) on the carrier branded android devices than on the Cupertino-controlled price of the iPhones...

 

[s.m.knipe]

Edit: yup. one women was actually yelling because verizon hires people that don't like the iphone, and that everyone should like the top selling device, otherwise people are going to be sold phones that they didn't come in for. she was upset because she bought a nexus, but her husband wanted her to get an iphone so he could play with it.

WOW. so basically she brought it home and he told her it was junk and make her feel stupid. man I'm glad I'm not in retail anymore...

 

[Unforgiven]

So if that chart is to be held as a source of comparison between iOS and Android, iOS 5, the latest iOS, has had more crashes than all of Android combined. And I believe Android still has more market share than iOS?

Wow...simply wow.

You know I am a huge Android fan, so I am no iPhone troll. To be fair to the survey, and iOS, it was conducted just prior to the GNex release here in the US (highlighted by the fact the the only ICS build is 4.0.1). I haven't read the whole article yet, going there next, but if the data was collected in the US only, that would mean that this would reflect only custom builds or leaked roms for ICS. New OS = App errors.

 

[EarlyMon]

I don't know about other carriers, but Sprint had to commit to move specific numbers of the iPhone product to get the franchise.

I can't imagine a way for corporate marketing to shoot itself in the foot and hit penalties if that's a common contract.

But I can imagine Apple wanting to do spin control on this, and wanting very badly for this to not go viral.

Tweet your friends, because it deserves to go viral.

I still maintain that there is no perfect phone, neither Android nor iPhone.

But I will also insist that an informed consumer is an empowered consumer. If the myths here are incorrectly accepted as facts, consumers deserve to know.

 

[EarlyMon]

Platform Versions | Android Developers

ICS is at an estimated 1% of Android market penetration.

 

[s.m.knipe]

I don't know about other carriers, but Sprint had to commit to move specific numbers of the iPhone product to get the franchise.

I can't imagine a way for corporate marketing to shoot itself in the foot and hit penalties if that's a common contract.

But I can imagine Apple wanting to do spin control on this, and wanting very badly for this to not go viral.

Tweet your friends, because it deserves to go viral.

I still maintain that there is no perfect phone, neither Android nor iPhone.

But I will also insist that an informed consumer is an empowered consumer. If the myths here are incorrectly accepted as facts, consumers deserve to know.

very true

 

[s.m.knipe]

You know I am a huge Android fan, so I am no iPhone troll. To be fair to the survey, and iOS, it was conducted just prior to the GNex release here in the US (highlighted by the fact the the only ICS build is 4.0.1). I haven't read the whole article yet, going there next, but if the data was collected in the US only, that would mean that this would reflect only custom builds or leaked roms for ICS. New OS = App errors.

wouldn't those leaked or custom ROMS (with a higher probability of instability) over-inflate ICS' app crash numbers, benefiting the iOS numbers by comparison?

 

[EarlyMon]

wouldn't those leaked or custom ROMS (with a higher probability of instability) over-inflate ICS' app crash numbers, benefiting the iOS numbers by comparison?

And that's exactly what the numbers show - but it doesn't help any.

ICS is just over 1% of crashes during the study period, for all devices, and given that ICS is at 1% of the Android share today, and was likely at about 0.5% tops during the study period, and normalizing for the Android numbers, this yields roughly a 5%(*) failure score against devices occupying only 1/10 of that sample set.

This clearly suggests a higher likelihood of ICS app failure. And given that all apps had not been sanitized for ICS (where needed) and not ready for newer hardware (where needed) and in those terms, your explanation is as valid as any.

However -

It still doesn't change the overall comparison when considering the gross magnitude of difference between Android and iOS concerning app failures.



* footnote on normalization - Android failure occupies about 20% of the graph, ICS, 1%. 1 part in 20 is 5% and seems close enough for rock and roll to me.

 

[Unforgiven]

wouldn't those leaked or custom ROMS (with a higher probability of instability) over-inflate ICS' app crash numbers, benefiting the iOS numbers by comparison?

The point I was trying to make was that the survey ended on 12/15, the 1st day the GNex was available in the US, so the number of users that could potentially report crashes in ICS was potentially very small prior to that (at least in the US). I am going to over exaggerate to make my point here, and I'm not trying to flame.

If there were 1 billion iOS 5 devices in the wild and 0.5% of them had one app error it would still be 5 million app errors reported. If there are 100 Android 4 devices in the wild and 100% reported 10 app errors it would be 1,000 errors. If you put those two statistics in a chart like the one linked below, you could see results like you see between iOS 5 and Android 4.

BTW, I recommend reading the actual article on Forbes. It is very interesting. s.n.knipe, this comment isn't meant to imply that you haven't read the article, its a general comment for anyone reading the thread.

edit: Earlymon is saying it way better than I am just above me ^^^^^^^^^^^^

 

[AntimonyER]

A couple things to consider/discuss.

One, widgets make Android app utilization less than Apple. I don't need to go into a weather app to see basic weather info, or my calendar, etc. So that is one thing that the apple folks can point out as skewing the numbers slightly, since they need to use apps more often (although they have to admit Android presents info quicker in order to make this argument )

Second, multi-tasking is handled differently between Android and iOS. I am not 100% up to date exactly how iOS 4.0+ handles multi-tasking. Would that affect how we should look at these numbers. I am wondering if the app closures per launch is even comparable. If an app crashed once a week, on android, that could theoretically be a 1-1 rate, if the app had been running the entire time. If iOS is psuedo-multitasking, and still closing the app down and restarting when it is called, there would be a lot more launches compared to android.

 

[dylo22]

One factor the article doesn't account for is hardware. iOS is suppose to be able to run on iphones that are several generations back (I think for ios5 the bare minimum is 3GS). And from what I have read the 3gs is barely capable of running io5 and lots of people are complaining about slow downs, crashes and lock ups. So I'm wondering if a good percentage of ios5 crashes in that chart has to do with ios5 running on older hardware such as 3gs or maybe even 4g.

 

[EarlyMon]

One factor the article doesn't account for is hardware. iOS is suppose to be able to run on iphones that are several generations back (I think for ios5 the bare minimum is 3GS). And from what I have read the 3gs is barely capable of running io5 and lots of people are complaining about slow downs, crashes and lock ups. So I'm wondering if a good percentage of ios5 crashes in that chart has to do with ios5 running on older hardware such as 3gs or maybe even 4g.

I would submit that many with lesser Android hardware try to over-upgrade and report similar results. I have seen some people of these forums not like an update and wanting to regress - but nothing of the magnitude as happened with iP3gs owners.

Add to that that Apple had a record Q4 2011 with 37 million units shipped, and without a doubt, those new device owners were downloading and trying apps during the holiday season.

The Apple mantras are: just works, easy to use, Apple-ensured updates, and no fragmentation.

Even if what you submit is absolutely true and there are no mitigating factors as I suggest, the numbers say the mantras are not entirely accurate.

 

[jerofld]

You know I am a huge Android fan, so I am no iPhone troll. To be fair to the survey, and iOS, it was conducted just prior to the GNex release here in the US (highlighted by the fact the the only ICS build is 4.0.1). I haven't read the whole article yet, going there next, but if the data was collected in the US only, that would mean that this would reflect only custom builds or leaked roms for ICS. New OS = App errors.

While I can see the Galaxy Nexus having an impact on the numbers, I don't think that it would make that much of a difference. At best/worse (depending which way you look at it), I bet it would make all of Android equal to iOS 5 alone. Of course that's pure speculation, but I think it's a fair estimate

But you have a good point. If the chart was re-polled again now, the ICS numbers would probably been increased.