The best advice anyone can give you about following the polling for the presidential race is…just don’t. For one thing, with six months to go, it’s still way too early. And for another, even when we get close, the various normal standard polling errors are probably going to be larger than the margin between the candidates, at least if it’s a close election as most expect. And if it’s a blow-out, you won’t have to keep hitting refresh at the polling sites to know it.
That said: I know you’re going to do it. You know you’re going to do it. Hell, keep saying not to, and I’m doing it. Yeah, our time on politics might be better spent making phone calls for the candidates we like or lobbying elected officials or at least learning something about policy but the truth is most of us want to know the outcomes of elections and we want to know now.
And polls look so real.
Indeed, they are real. They just aren’t always predictive, and they’re almost never perfectly predictive. As with all information, they can be interpreted well or badly. So here’s a bit of guidance as you and I, mistakenly, keep looking at the polls.
I think folks know this by now, but can’t say it often enough: Averages, not individual surveys. It’s not just that individual polls can be off in all sorts of ways; it’s that we simply aren’t equipped to pay the same attention to the ones that look good for our candidates and the ones that look bad. None of the published aggregators are perfect, and alas Pollster is long gone, but the 538 polling average is solid and you can also use RealClearPolitics or others.
In the old days - before the 2016 election - I would have said to only look at national polls, not state polls, at least until the final few weeks. And now? Really, that’s still my advice.1 Just keep in mind that if things are similar to the last two cycle that Democrats will need to win by 2, 3, or even 4 percentage points in order to prevail in the electoral college.
Why not just focus on the handful of swing states? For one thing, there often just aren’t enough surveys in those states to establish meaningful averages. The folks at 538 have logged 8 different Pennsylvania polls since the start of April, but a bit more than 50 national ones. It’s also typically been the case that while some of the work is excellent, overall the quality of national polls is probably still better than state polls - and there’s a better chance that the aggregators will learn to exclude those that are really lousy.
What’s more, we just don’t know enough to know which contested state will be the tipping point, and therefore once you go down that road you’ll wind up updating yourself on three, five, or more states. And don’t kid yourself: You’ll be looking at the national numbers too. You really don’t want to do that to yourself.
The last piece of advice: Ignore the cross-tabs. Yes, really. Unless the poll you’re looking at was designed to study a particular sub-population, the chances are just too high that anything startling that shows up is actually either a random result or an artifact of how the data were gathered.
I suppose I should explain that…you probably know that if you flip 5 honest coins you’ll occasionally get 5 heads, but if you flip 1000 honest coins you’ll never get 1000 heads. Right? Just how luck works. Think about the cross-tabs as, say, the years that those coins were minted. Or even worse: Maybe you have a bunch of difference coins, and so you’re now looking at only, say, dimes from 2006. And quarters from 2017. And pennies from 2022. And pennies from 1998. The more sub-categories you look at, the better the chances that one of them comes up weird just from dumb luck.2
But it’s worse than that, because we can’t know whether we’re dealing with “honest” coins here - that is, whether the particular respondents from various groups and sub-groups that the polling outfit found are actually representative of their groups. The smaller the group, the more ways things can go wrong, and different firms are going to use different methods to handle that problem, some more successful than others. (For a more technical analysis, see Natalie Jackson here; unfortunately that one is gated, but for a couple more days you can read her excellent explanation of why polls vary here for free).
Look: I understand the urge to know what’s happening and why, down to the grainiest detail. It just can’t be done. Especially this far out from the election. And by the way: I’d recommend a ton of skepticism for pundits who try this, although some do include sufficient disclaimers and stick to careful studies that focus on subgroups and therefore have a better chance of avoiding big mistakes.
So: averages and not individual polls; national and not state surveys, at least until the last few weeks, but remembering that there’s a good chance that the electoral college will once again have a Republican bias; and lay off the crosstabs. Do that, and at least you reduce the risk of actively misinforming yourself. Hey, that’s something!
Of my three guidelines here, I think this is the one that would get the most disagreement from people I respect, so keep that in mind. Nevertheless, I’m pretty sure I’m right.
Note too: Go too broad in defining your groups, and you will be vulnerable to getting it wrong because of intra-group differences (so that if your Latino subgroup is all Florida Cuban-Americans it’ll tell you something different than if they’re all Los Angeles Mexican-Americans), but narrow it down to avoid that problem and the number of respondants in each category gets even smaller.