Archives for 2024

Here’s a free link to Josh Marshall’s members-only article on polling.

I’m glad he has the patience to write this all out because all I can personally muster is to say that the polls have gotten to be way more black magic and secret sauce, which I sometimes shorten to “ignore the polls, they’re more bullshit than science at that point, just do the work.”  Because I seem to be out of patience at the moment.

But lucky for us, Josh Marshall isn’t!

I wanted to share with you a few ideas, possible insights and caveats about campaign polls. These aren’t original to me in most cases, just some general points, observations, etc.

First, herding: Herding is the phenomenon in which even professional, good-faith polling operations start grouping together in the latter stages of a campaign because you don’t want to go too far out from the consensus numbers. Right now the national top lines have been between 2 and 4 points in Harris’ favor for a couple months. If you do a poll that gets you plus 10 in either direction, you’re going to think or are liable to think something’s wrong with your numbers. Somehow you’ve just got a spoiled set of data. Maybe you don’t release that poll or maybe you look again at the numbers and decide there are too few of some demographic subset and you re-weight that and it brings the topline back close to that 2-4 range.

It’s also the case that voter choice gets more settled in the final weeks of a campaign. So maybe the voters are actually herding themselves. There are lots of possibilities. But the general point here is that there are factors which can drive even ethical and professional pollsters in this herding direction.

It goes without saying that the more fly-by-nightish operations probably do this a lot. It’s sort of assumed that a lot of the junk pollsters let their freak flags fly for most of election season but then try to bring them in line with the herd in the final weeks to protect their reputations.

Implicit in the herding discussion above is what I’ve noted several times recently: the stability of this race is one of its most defining features. That could just be how settled the public mind is on this choice. It must be some of that. But I’ve heard another possibility proposed recently: Maybe pollsters are now modeling so aggressively that they’ve wrung all the movement out of the numbers?

Remember in earlier discussions I’ve noted that a poll is based not just on the 500 to 1,000 people polled but also on the pollster’s theory of the electorate. This is the art part of the art and science of polling. Modeling has become more aggressive as non-response has become a bigger and bigger problem. So you’re not just weighting men and women and Black people and white people but affluence, education levels, city vs. rural, how people say they voted last time, perhaps party identification. At a certain point you’re basically just saying you need 49% Trump supporters and 49% Harris supporters. I jest of course. But there could be something to this: maybe the race is so stable because the variability has been squeezed out through aggressive modeling. No one is proposing this as more than an idea, a “what if.” But it’s worth having in the back of our minds.

Next there’s an article you should read by Nate Cohn of the Times. It gets at a key methodological question about this year’s polling. Historically, it was always considered bad practice to weight polls based on how voters say they voted in the last election. Basically, people don’t always remember or tell the truth about how they voted. And it tends to pump up the advantage of the party that lost the last election. In other words, by traditional standards and accepted methodological considerations, by adding this voter recall weighting the pollsters have built a pro-Trump advantage into every result. But in an effort to more effectively avoid missing hidden Trump voters, most but not all pollsters decided to weight for voter recall for 2024. This isn’t obviously a bad idea, as Cohn explains. Polls did underestimate Trump in 2016 and 2020. And there are various ways pollsters have tried to account for the reasons why traditionally this isn’t a good idea to do. Cohn’s point and one I think he’s right to make is that that decision leads to two very different stories of the election. I’ve argued here and on the podcast that there’s as good a reason to think pollsters over-adjusted after 2024 as there is to expect Trump to again outdo the polls. Depending on how the race plays out it might be this specific decision that explains why one group or another of pollsters missed the mark.

Open thread.

show full post on front page

Josh Marshall Has Some Thoughts about PollsPost + Comments (153)