One Giant Leap: Understanding the Rapid Evolution of Smartphones for Market Research

It’s no surprise to most in the market research industry that mobile smartphones are swiftly taking over the mobile phone market, with Nielsen’s Mobile Insights group reporting that smartphone owners in the US make up 43% of the total market as of Q3 2011. This number is seemingly in a constant upward climb with no end in sight.  Most estimates predict that smartphones will overtake feature phones sometime in Q2 of 2012. 

Given this rapid adoption of mobile smartphones in the US and worldwide, it is imperative to continue to learn how people are using phones and what standards and best practices should be considered prior to shifting to smartphones for survey data collection. This necessitates rigorous testing of the smartphone mode of data collection, to best understand how to optimize the experience for users, and, thus increase the likelihood of achieving high response rates and accurate data.

Further, smartphones themselves are rapidly changing and improving in size, clarity, and ease of use.  For example, the Samsung Blackjack phones like that used by Peytchev & Hill (2010) in their seminal research on mobile web survey design are quickly becoming extinct.  Those phones had a screen size of 2.2” and used a keypad physically located below the screen. According to NPD Group (2011), today’s top-selling smartphones are the Apple iPhone, HTC Evo, and Motorola Droid 3, all of which have displays of 4” or more and use touchscreen functionality, not to mention higher resolution displays than phones prior to 2009.  Although the growth of smartphone penetration is not slowing, the overall optimal form factor appears to have been reached. What is known is that the torrid evolution of smartphone technology has changed the way people use mobile phones, with more data being consumed in 2011 than ever before. 

Thus, Nielsen designed a test to address these issues and assess survey design on smartphones compared to the "traditional” online survey mode. The purpose of this research was to 1) provide a direct comparison of survey responses taken on a mobile device vs. online, 2) test common survey question types often used by market researchers, and 3) better understand the effects of creating an "optimized” mobile survey for respondents.

Key Findings:

- More smartphone owners are using their devices as web browsers than ever before. Nielsen Mobile Insights and other research groups have provided evidence that mobile data usage is at all-time highs, but we unexpectedly found that, even when respondents were sent an email asking them to complete the survey online, over one-quarter chose to complete the survey on a non-optimized web browser.  This means that not only did these respondents use their smartphone to check their email, they also selected the link and completed the survey in that moment.  Additionally, they were willing to deal with the non-optimized screen even though it took them on average 3 minutes longer to complete the survey than if they had taken it on a PC.  This speaks to the way smartphone owners are using their phones and how we as market researchers can leverage mobile surveys.

- The mobile platform used by mobile respondents affected response. Because of our large sample size, we were able to examine differences in response times for various mobile platforms.  Consistent with national averages, the majority of our respondents owned either an iOS or Android device.  While, overall, iPhone users took 11 seconds longer to complete the survey, the open-ended items were completed much faster by iPhone users than Android users (and the former typed more characters in less time).  When comparing the two platforms, it should be noted that most Android devices do not have autocorrect features while this is the default setting for iPhone.  This is one possible explanation for the difference on open-ended items.  Regardless, survey designers should consider this when creating a survey.  Even though Android users did not skip these questions, they appear to have provided less rich responses.

- Mobile surveys have many characteristics of survey modes that preceded it.  We embedded classic experimental survey questions within the questionnaire in order to compare the mobile mode with online or paper and pencil surveys.  Two phenomena discovered in online and paper and pencil modes were also replicated in mobile mode.  First, we manipulated frequency  scales while keeping question content consistent.  Consistent with other survey modes, mobile users indicated higher consumption when given the higher frequency scales.  Secondly, we showed that, when given a large text box versus a small text box, respondents will provide longer responses, a phenomenon also observed in other modes.  These two findings suggest that many of the best survey design practices utilized for online surveys should be used for mobile as well.

To view the complete paper (PDF) with methodology section, please click here.

By Justin Bailey & Thomas Wells

Justin Bailey leads the digital side of Nielsen’s Research Methods group. He is also a Founding Board Member of the Mobile Marketing Research Association (MMRA). His work is focused on testing new technologies in mobile and online. Nielsen has many uses and applications for mobile data collection, and the Research Methods teams collaborates with other Nielsen groups to understand how to leverage these innovative, cutting-edge methodologies. Justin’s primary research focus has been on ways to effectively capture data and reduce respondent burden using mobile devices both in the US and internationally.

Tom Wells leads the online side of Nielsen’s Research Methods group. His current research interests include online survey methodology, mobile survey experiments, and Facebook user research. Tom designs and manages innovative methodological studies and provides advice and recommendations on study design and questionnaire design to other Nielsen groups.