Pros and Cons of Adding Cell Phones to Telephone Samples

Should you include cell phones in your telephone samples?

The increase of cell phone-only households and the use of cell phones most of the time, even by those who still have landlines, is becoming a serious issue market researchers involved in survey research can’t overlook. Cell phone samples need to be included in the sampling plan.

Why can’t we ignore cell phones? It is a matter of coverage, as Scott Keeler, the Director of Survey Research at the Pew Research Center pointed out in a recent presentation to members of the Insights Association (former MRA). He called attention to the fact that at least 25% of adults can be reached only on a cell phone, a proportion that is even higher in certain groups such as young adults 25 – 29 (51%), 18 – 24 (40%) and Hispanics (34%).

This means that when we rely only on landline sample frames, certain population segments are likely to be underrepresented. Analysis of data from the Census and Pew Research Center surveys shows that only 7% of adults 18 -29 can be found in landline samples, while 66% are adults over 50.

Including cell phones in telephone samples is more complex than just calling landlines. Keeler pointed out a set of practical, legal, and ethical issues we face when dealing with cell phone samples.

Legal

• Because of the Telephone Consumer Protection Act (TCPA) established in 1991, landline and cell phones are separated, so researchers wanting to include cell phones in telephone samples have to work with a dual-frame approach.
• The TCPA also prohibits automatic dialing to cell phones unless there is prior consent from respondents, which means cell phone numbers need to be hand-dialed, making it very inefficient and costly. Landline numbers that have been transferred to cell phones need also to be treated as cell phones, which requires some phone number scrubbing.
• Callbacks to increase response rates are also limited. There is a high likelihood that the same person answers a cell phone, so too many callbacks can be perceived as harassment.

Ethical

• Many teenagers and children have cell phones and they are more willing to answer a call, so there is a higher risk of dealing with minors. This tends to result in more ineligible (and larger lists) cell phone numbers that need to be dialed).
• Often people move and keep their cell phone’s area code so the geographic information associated with cell phones may be wrong which complicates the time of day when a call may be appropriate.  This can results in that someone with a West coast area code living on the East coast, may get a call for a phone survey a little too late for his or her like.
• We also need to consider respondents’ safety at the time we call them since they may be driving or doing something in which if they get distracted it may endanger them.
• Respondents’ privacy is another issue to take into account. We may be asking sensitive questions respondents don’t feel comfortable answering if they are in a public area or close to people with whom they don’t want to share certain information.

To deal with these ethical issues, telephone surveys with cell phone samples need to be designed to carefully screen respondents and verify to whom we are talking, if it is the right time to call if it safe for the respondent to talk, and if the person is in position to talk about sensitive topics.

Data Quality

Concerns about the quality of the data for cell phone samples have also been voiced. Audio quality during the interview, potential distractions for the respondent, respondents’ multitasking, possible desire to rush to complete the interview, and sensitive topics are among the factors that could have a negative impact on data quality in cell phone samples.

How Much Does it Cost?

Unfortunately, including cell phones in telephone samples doesn’t come cheap due to the inefficiencies brought by the above mentioned legal issues and the pricing structure of cell phone services in the US, where consumers pay not only for the calls they make but also for the ones they receive.

According to the American Association for Public Opinion Research (AAPOR)‘s Cell Phone Task Force, cell phone interviews are typically at least twice as expensive as landline interviews and can be three or four times more costly if cell phone-only samples are used.

To find guidance in how to meet the difficult and costly challenges of cell phone surveys check the 2010 AAPOR Cell Phone Task Force Report: New Considerations for Survey Researchers When Planning and conducting RDD Telephone Surveys in the U.S. With Respondents Reached via Cell Phone Numbers

What Are the Alternatives?

• Online panels: These are often used in the market research industry due to their ability to provide affordable samples, albeit most offer non-probability samples. They may provide coverage of some of the cell-only population, but there are concerns about data quality and representativeness.
• Address Based Sampling (ABS): These are household samples drawn from USPS Delivery Sequence File (the “DSF”) which can be used for telephone, Internet, mail, or in-person interviewing. ABS covers 98% of the households in the US and 60%-70% can be matched to a phone number. There seems to be a growing use of this approach by big polling entities and survey cost could be comparable to RDD samples, but more expensive than online panels.

 Finally, if your organization can only afford landline samples, luckily, according to Keeler:

•  Most analyses have found no differences in data quality between cell phone and landline samples.
• There is some bias in coverage in landline-only samples when compared to samples that include cell phones, but the bias is not too big. In an analysis done by the Pew Research Center for several of their surveys, they found, as the chart below shows, that most of the time the difference between both sample frames was 1 or 2 percentage points. Its relevance depends on what we are measuring. If we are talking about polls predicting an election, a small bias can be of great significance, but it is possible that for many commercial research studies, it doesn’t have the same impact.
• Response rates in landline samples are similar to that of cell samples, and both have a similar level of respondent cooperation.

 Although these are good news, they don’t provide a solution to the coverage issue, so if you are interested in reaching young groups that can’t be reached by landlines, and can’t afford a cell phone sample, you should consider a hybrid approach that combines telephone and online surveys or give ABS a try.