Sunday was a bad day for pollsters, adding yet another poor performance following a string of recent high-profile polling misfires. This time the final round of polls on the Greek referendum showed the bailout deal failing by 3 to 4 percentage points, but the actual vote was not nearly as close. The "no" side prevailed by a 22-point margin -- 61 to 39 percent. To be fair, polls on the referendum faced an unusually high degree of difficulty. Greek Prime Minister Alexis Tsipras called for the special vote just eight days before it occurred. In between, Greece failed to meet an International Monetary Fund deadline, effectively putting itself into default and creating a genuine crisis. The lengthy, technical ballot language added another element of complexity. All together, Greece's pollsters had "some pretty good excuses," as Nate Silver put it. Although definitive conclusions about why the polls missed in Greece will come only after the pollsters thoroughly investigate their methods and data, there are signs that the same challenges facing survey-takers elsewhere may have made a bad situation worse in Greece. Here are some reasons why the polls in Greece erred: A late "no" trend? The public polls showed considerable volatility over the eight days between June 27, when Tsipras announced the referendum, and July 5, the day of the vote. The predicted "no" vote dropped significantly for five days before nudging back upward during the last 48 hours. But that slight "no" trend at the end cannot explain why the polls deviated so widely from the actual results, especially since the four conducted on Election Day still understated the size of the "no" vote. Ultimately, the "yes" position received 39 percent of votes cast, far less than the estimates of more than 20 different polls conducted over the last few days of the campaign. A hidden "no" vote? Researchers with experience in ballot referendums believe that polling often understates the number of "no" supporters -- usually, those who will vote for the status quo -- in such situations. Two highly relevant examples are the sovereignty votes in Quebec in 1995 and Scotland in 2014. In both cases, people cast far more "no" votes than polls had forecast. In both cases, those who had been undecided in earlier polling appeared to swing decisively toward "no" in the final hours of the campaign. That pattern led Canadian survey researcher Claire Durand to speculate that a "systematic bias" might exist that pollsters "will have to take into account in future referendum votes." Yet simple math tells us that the full "no" vote could not have been hiding only among those who said they were undecided. Even if every undecided respondent went on to vote "no," poll results still exaggerated the "yes" vote. Confusing ballot wording: Another potential problem came from the ballot wording. In the U.S., most pollsters try to replicate ballot language as closely as possible in their questions, and they struggle when that language is lengthy or confusing. Most of the polls in Greece asked fairly straightforward questions, such as, "How will you vote in Sunday's referendum on whether to ratify the agreement that Greece's creditors offer" (as translated by the Bloomberg/Macedonia Research Institute poll). The actual question that voters considered was far more complex (as translated by The New York Times): Should the deal draft that was put forward by the European Commission, the European Central Bank and the International Monetary Fund in the Eurogroup of June 25, 2015, and consists of two parts, that together form a unified proposal, be accepted? The first document is titled “Reforms for the Completion of the Current Program and Beyond” and the second “Preliminary Debt Sustainability Analysis.” "For those who can't read Greekdebtspeak, well, you're on your own," the BBC warned its readers after providing its own translation. The language looked "borderline surreal" to University of Chicago political scientist Stathis Kalyvas, who also noted the "irregularity" of putting the "no" option before the "yes." Pollsters are understandably reluctant to read lengthy paragraphs to respondents over the telephone, but it is possible that questions closer to the actual ballot language might have obtained a more accurate result in this case. Missing cell phones: Although most of the Greek polls were conducted by telephone, none of the public reports examined by The Huffington Post (in both Greek and English) made explicit reference to calling both landline and mobile phones. That doesn't mean that no pollsters were calling both landline and mobile, but those who failed to sample mobile-only households risked understating the "no" vote. While 77 percent of Greek households have a landline phone, 18 percent are accessible by mobile phone only. According to the 2014 Eurobarometer, the European Commission report that compiled these statistics, "the main socio-demographic differences relate to age. The younger the respondents, the more likely they are to have only mobile telephone access." The problem is that younger voters were apparently far more likely to oppose the bailout package on the ballot. Results from the Public Issue poll, re-weighted after the election to match the overall vote results, showed 85 percent of those aged 18 to 24 voting "no," compared to just 45 percent of those over 65. That pattern alone cannot explain the polling misfire. Pollsters who failed to sample cell phones were likely weighting by age -- i.e., adjusting their data -- to account for the missing younger voters without landlines. In practice, that would entail giving greater weight to younger voters who were reachable via landline. Thus, the bigger issue is whether the younger voters with landlines had opinions different from those of their mobile-only counterparts. Voters who didn't respond: The 2012 U.S. presidential election yielded circumstantial evidence that varying response rates can affect polling estimates. Following President Barack Obama's lackluster performance in the first debate in early October, his support fell dramatically in national polls. Many showed Republican challenger Mitt Romney pulling slightly ahead, but Obama regained a slight lead before Election Day and ultimately received a greater share of the vote than most of the national polls predicted. Research conducted among voters who were interviewed before and after the debate, however, showed few had actually changed their minds in this period. The seemingly dramatic shifts were due mostly to a difference in the willingness of partisans to be interviewed -- Obama supporters were briefly less willing to participate in polls after the first debate. Did something similar happen in Greece last week? The initial surveys showed opposition to the bailout deal that came close to the final vote, but heavy television news coverage of the growing crisis, particularly once Greece defaulted on Wednesday, may have caused "no" voters to take a step back. Events may have led to true volatility in public opinion, but polls would have exaggerated that volatility if "no" voters were suddenly less willing to answer pollsters' questions. In addition, as a general rule, survey response rates are lower among younger people and those with lower incomes, exactly the voters most likely to vote "no" in Greece last Sunday. Roula Nezi, a political scientist who has worked with many survey researchers in Greece and across Europe, noted that lower response rates are also typical among those hardest hit by austerity measures. "This is something we've observed in all bailout countries," she said. At very least, some saw a possible voter backlash against apparently one-sided media coverage favoring the bailout proposal. "TV media's very open support of the YES vote seems to have discouraged them from supporting what they [the media] were suggesting," political scientist Iannis Konstantinidis told The Huffington Post via email. "Mass media are regarded as pillars of the old party system in Greece and therefore whatever they are saying is discredited." Herding? Nate Silver sees evidence that polls "herded" toward a consensus in Greece. "Whereas the first few polls were all over the place," he wrote, "the polls began to track one another closely by a few days into the campaign. And all polls conducted on election day itself showed nearly identical results, each with 'no' ahead by 3 or 4 percentage points." Whatever the full explanation for the misfire, Silver is right that conditions faced by pollsters in Greece were "a recipe for a polling disaster." Pundits should heed the lesson and exercise more caution in interpreting polls, especially in such a volatile environment. Janie Velencia and Thanos Kampylis contributed reporting. -- This feed and its contents are the property of The Huffington Post, and use is subject to our terms. It may be used for personal consumption, but may not be distributed on a website.
 |
| READ THE ORIGINAL POST AT feeds.huffingtonpost.com |
