Google says Androids receive 58% fewer scam texts than iPhones, but I have a few questions

If you’re receiving more scam text messages than your friends and family, it turns out it could be your mobile platform, at least according to a recent survey from Google and YouGov. According to Google, Android users report receiving 58% fewer scam text messages than iOS users, and that number is even better on Pixel devices.

This data comes from a survey conducted by Google and YouGov of more than 5,000 Android and iOS users in the United States, India and Brazil. The data indicates that Android users as a whole were “58% more likely than iOS users” to report that they had received no fraudulent text messages in the week preceding the survey, and that iOS users were 65% more likely than Android users to report receiving three or more fraudulent text messages in a week. Overall, it also indicates that Android users were 20% more likely to describe their device’s scam protection features as “very effective” or “extremely effective.”

Looking specifically at Pixel versus iPhone, the survey also found that Pixel users were 96% more likely to report no fraudulent texts than iPhone users, and iPhone users were 136% more likely to say they had received “a high volume of fraudulent messages” than Pixel users, as well as 150% more likely to say their device was “not at all effective at stopping mobile fraud.”

These numbers don’t look good for Apple, although it’s worth pointing out a few details before taking Google’s word for it.

What is YouGov?

First, it must be recognized that Google did not conduct this study alone. Although the company hasn’t detailed its partnership with YouGov here, YouGov itself is a respectable entity, widely cited in market research and even political news. The organization specializes in surveys and has been cited in numerous articles, from the New York Times to Lifehacker’s sister publication Mashable.

It’s also worth pointing out that YouGov wasn’t the only independent entity Google worked with when putting together its report. The company also contacted Counterpoint Research to evaluate its AI-based anti-scam protections, and Leviathan Security Group to evaluate scam protection on four Android phones. The company used these studies to create infographics illustrating its security features compared to those of the iPhone, which you can see below.

Credit: Google

Credit: Google

Note, however, how Google has put the information together here. Specifically, this implies that the iPhone offers no protection against email scams, while ignoring that many iPhone users access their email through the same Gmail app as on Android. It also doesn’t mention Safari’s ability to block pop-ups or the iPhone’s ability to screen phone calls, which makes me think Google might be picky about how it presents data here. As such, one should also examine the results of one’s investigation with a critical eye.

Questions about methodology

While Google’s survey results are likely accurate for the audience surveyed, I have some questions about the methodology.

First, there’s the problem that this is a survey rather than a more impartial look at text message inboxes. While useful for gaining insight into how users view their devices, surveys are sensitive to respondents’ words, meaning they can be influenced by respondents misremembering details or misunderstanding questions.

The biggest question for me, however, is the sample size. While I applaud Google for conducting its survey in three different regions, the company didn’t specify how many of its respondents owned iPhones versus Android devices. It’s possible that iPhone users have reported more scams simply because the iPhone is more popular in the United States, for example. To be fair, these numbers are reversed in India and Brazil, but without knowing how many people Google and YouGov surveyed and where, it’s hard to know if the data is affected by other factors.

For example, in addition to potentially being more numerous depending on the distribution of the surveyed population, iPhone users might also be more likely to fall victim to fraudulent SMS messages simply because scammers might want to target this user base more, as was once the case with Windows viruses. The iPhone is known as a luxury device, implying that its user base might have extra money to spend, making it an enticing target.

Can you believe Google’s survey?

So, can you take Google’s surveys at their word? Yes and no.

With the help of YouGov, it is likely that the results presented are true, although they may still be misleading. The only way to know for sure would be to look at a detailed analysis of the survey itself, but unlike assessments from Counterpoint Research and Leviathan Security Group, there doesn’t appear to be a way to review the raw data from the survey at this time, with only Google’s blog reporting its findings.

I contacted Google and YouGov independently to ask about their methodology, and did not immediately receive a response. For good measure, I also reached out to Apple for feedback and am still waiting for a response.

In the meantime, I would take this investigation with a pinch of salt. It’s almost certainly based in truth, but it’s important to remember that it’s always part of a larger advertising narrative.

Google recently made a big deal out of calling Apple’s ecosystem a walled garden. If the company can convince users that it can offer them greater freedom while improving their protection, that would be a big victory. This, however, coincides with Android’s efforts to shut itself down in the name of security, so perhaps the truth is that the two platforms aren’t so different after all.