Posts Tagged ‘Sam Wang’

On Polls

Saturday, November 5th, 2016

Election season is upon us which means that poll-watching season is upon us. Back in 2012, I wrote a long post about the analysis of the polls. Specifically, I focused on the 2000 election in which Bush led the polls going in, Real Clear Politics projected a Bush landslide and … it ended in a massive recount and a popular-electoral split. I identified the factors that I thought contributed to this:

In the end, I think it was all of the above: they overestimated Nader’s support, the polls shifted late and RCP had a bit of a bias. But I also think RCP was simply ahead of its time. In 2000, we simply did not have the relentless national and state level polls we have now. And we did not have the kind of information that can tease out the subtle biases and nuances that Nate Silver can.

Of course, I wrote that on the eve of the 2012 election, where Obama significantly outperformed his polls, easily winning an election that, up until the last minute, looked close.

The election is now three days away which means that everyone is obsessed with polls. But this year, a split has developed. Sam Wang is projecting a 98% chance of a Clinton win with Clinton pulling in about 312 electoral votes. HuffPo projects a 99% chance of Clinton winning the popular vote. Nate Silver, however, is his usual conservative self, currently giving Clinton only a 64% chance of winning. So who should we side with?

To me, it’s obvious. I would definitely take Silver on this.

Put aside everything you know about the candidates, the election and the polls. If someone offered you a 50-to-1 or a 100-to-1 bet on any major party candidate winning the election, would you take it? I certainly would. I would have bet $10 on Mondale in 1984 if it was a potential $1000 payoff. And he lost by 20 points.

It seems a huge stretch to give 98 or 99% odds to Clinton, considering:

  • Clinton has never touched 50% in the poll aggregates.
  • There are still large numbers of undecideds and third party supporters who will doubtless vote for one of the two candidates (and Trump’s recent surge has come from fleeing Johnson voters).
  • We have fewer live interview polls now than we did in 2012.
  • As Nate Silver noted, the average difference between final polls and the election has been about two points.
  • Basically, I think Wang and HuffPo are not accounting enough for the possibility that the polls are significantly off. In the last 40 years, we’ve had one Presidential election (1980) where the polls were off by a whopping seven points. That’s enough for Trump to win easily (or for Clinton to win in a landslide).

    Moreover, Wang’s and HuffPo’s results seem in contradiction to each other. If Clinton really did have a 98% chance of winning, wouldn’t you think she’d get more than 312 electoral votes? That’s the kind of certainty I would expect with a pending landslide of 400 or 500 electoral votes. A 42-electoral vote margin of errors is *really* small. All you would need is for the polling to be wrong in two big states for Trump to eek out a win (note: there are more than two big battleground states).

    This brings me to another point. Pollsters and Democrats have been talking about Clinton’s “firewall” of supposedly safe states that guarantee a win in the electoral college. But that firewall is a fantasy. When Clinton dipped in the polls in September, suddenly numerous blue states like Pennsylvania and Michigan were in play. And, in fact, Silver projects a bigger chance that Trump wins in an electoral-popular split than Clinton because many of his states are safer. The talk about a “firewall” is the result of people becoming drunk on state-level polling. We have 50 states in this country. Statistically, at least one should buck a 98% polling certainty. There are only twenty states that Real Clear Politics rates as “leans” or “tossup”. Statistically, at least a couple of those should buck the polling.

    Here’s another way of thinking about it. There have been 56 elections in American history. If Clinton really were a 98% or 99% favorite, a Trump would be the biggest upset in American electoral history. I find that claim to be absurd. Bigger than Dewey and Truman? Bigger than Polk’s election? Bigger than Kennedy’s? Bigger than Reagan turning a close race into a blowout?

    I should point out that having long tails of probability also means there is a greater chance of a Clinton landslide. That’s possible, I guess. But, admitting to my priors here, I find a Trump upset more likely than a Clinton landslide. Clinton is deeply unpopular with large parts of the country. She’s not popular with young people. Here in State College, Clinton signs and stickers are few and far between. This was not the case in 2008 and 2012, both of which were won handily by Obama. I really don’t see a Clinton landslide materializing, although I’ll cop to it if I’m wrong about that.

    Prediction is hard, especially about the future. I think a basic humility requires us to be open to the idea that we could be badly wrong. And 1-2% is way too small a value to assign to that. I think Clinton has the edge right now. But I would put her odds at more like 2-1 or 4-1. And I will not be shocked if Trump pulls this out.

    Because it may be a cliche. But there’s only poll that counts: the one taken on Tuesday.

    Update: One of my Twitter correspondents makes a good case that the variations in the polls are less reflective of changes in candidate support than in supporter enthusiasm. In the end, the election will come down to turnout — i.e., how likely the “likely” part of “likely voters” is.