Pew Research sees different polling results when cell phones are included

Posted on September 24, 2008

11


My wife’s younger siblings don’t have land lines. Several of my friends don’t have land lines either. Their cell phones are their only phones, and from what I’ve read, this is entirely normal for people my age and younger. So I’ve read claims that land line-only (LLO) polling is as accurate as land line plus cell phone polling with a bit if disbelief. And so it’s with great interest that I read a blog that hit my screen this morning from Future Majority.com that the Pew Research Center has identified a 2-3% shift in results in three Pew polls that were conducted twice each, one land lines only and the other land lines plus cell phones.

Polling organizations have been worried about the difference in results between the two different polling techniques for two basic reasons. The first is accuracy. As the Pew Research paper says, “U.S. government surveys estimated that about 15% of adults were ‘cell only’ in the fall of 2007 and the rate of increase since 2004 has been at least 2% a year, meaning that the number may be as high as 17% in this election cycle.” That’s a lot of adults who are being ignored in the polling data. Weighting factors to correct for statistically known differences between groups only work if the group being adjusted for has the same views on the poll as the groups actually polled. However, when the cell phone only group tends toward support for Obama over McCain, then the weighting factors used are simply wrong.

The second reason is cost. Calling land lines is cheap because polling organizations use computerized dialing programs. Using computerized dialing to dial cell phones is illegal, so polling organizations have to pay real people minimum wage or better to call each cell phone manually.

Michael Connery at FutureMajority.com is understandably concerned:

[I]f young voters are being underrepresented due to their cell-phone habits and likely voter screens, what will that mean for the accuracy of the polls leading up to election day? Could the pollsters be as wrong in 2008 as they were in 2004?

The unfortunate answer is yes, the pollsters could easily be that wrong. In fact, given that there are about 8% more cell phone-only adults in 2008 than there were in 2004, the pollsters could be significantly more wrong than they were in 2004.

It’s good to know that many polling organizations are including cell phones when they perform their phone surveys. But I haven’t yet found a list anywhere of which polling organizations include cell phones and which ones don’t. Knowing this information alone might make the difference between whether a poll should be taken seriously or not.

UPDATE: As per Gericault’s comment below, FiveThirtyEight.com has a list of the polls that are compensating for the cell phone effect, or have started doing so recently, and they are as follows: Pew, Gallup, USA Today/Gallup, CBS/NYT, Time/SRBI, NBC/WSJ, ABC/Washington Post, AP/GfK, the Field Poll in California, PPIC, and Ann Selzer. The following polls do not compensate for this effect, but may be “OK” due to other methodological factors: Internet-based pollsters, Zogby Interactive, Economist/YouGov, and Harris Interactive.

Ultimately, though, Pew’s response to this new data is the only correct response:

[T]he Pew Research Center for the People & the Press will include cell phone samples in all of its remaining election polls.

Advertisements
Posted in: Uncategorized