Why Political Polls are Useless

Now that the midterm elections are over, the Internet, pollsters and mainstream pundits are all agog. They are asking themselves endlessly, “How did we get it so wrong? How did the ‘red wave’ that we all knew was coming not materialize?”

I must confess to some schadenfreude in all this. After having read through far too many “horse-race” predictions, or at least having skimmed far too many useless headlines, I find it amusing to watch their authors pointing more fingers than they possess. But once my sardonic laughter fades, I have the sinking feeling that our media—virtually all of them—have done and are doing our nation, our voters and our democracy possibly irreparable harm.

For starters, let’s analyze how bad, really really bad, the recent polling was. It was abysmal in at least three big ways:

1. Skewed Samples. The old paradigm of bad political polling is now so famous it’s become a cliché: the Dewey-Truman presidential race of 1948. There the pollsters wrongly predicted Dewey’s win because they contacted voters only by telephone.

At that time, a much larger proportion of wealthy people, who voted Republican, had telephones than of working folk, who didn’t. So the choice of means to contact voters for polling skewed the polling sample toward the Republican candidate, Dewey. The result was that wonderful front-page photo of Harry Truman, as president-elect, grinning like a Cheshire Cat while holding up the false headline “Dewey Defeats Truman.”

So how could this happen again this year, 74 years later? Well, in 1948 voters’ availability to pollsters was a binary decision. They either had a telephone (a now-mostly-obsolete land line!) or they didn’t. The only other way to reach them would have been door-to-door canvassing—a method too expensive and complex for a national or statewide poll.

But today things are far more complex. We have at least five widely-used means of reaching individuals directly: land-line telephones, mobile phones, e-mail, text messages, and interactive social media like Facebook and Twitter. We also have a related phenomenon: incessant unwanted robocalls that plague both land-line and cell-phone users.

Did it ever occur to pollsters that many voters might screen out polling calls automatically or reflexively, seeing them as unwanted robocalls, and that their tendency to do so—not to mention their ability to do so automatically—might depend on their educational level, which recent polls have suggested is related to party affiliation?

2. Voter Reticence. It gets worse. Some voters just don’t want to talk about their electoral choices, let alone while they are still in the process of making them. And who might those voters be?

I can think of one whole category especially relevant to the election just past. How about young women of child-bearing age, especially those from evangelical or otherwise “conservative” homes? Might some or even most of them not want to reveal to a stranger their secret and personal desire to have abortion available to them, just as a backup? Might some not want to risk their parents, husbands or lovers overhearing them discussing the subject, let alone with a stranger?

Then there are the voters who make up their minds slowly and don’t want uninvited intrusion into their decision-making process. Is it impossible to conceive that those voters might be less willing to reveal their thoughts than the loudmouth who is already irretrievably convinced that the Demagogue is the answer to any problem and that “only he can fix it”?

The point here is not that this analysis is necessarily right. The point is that it, like all attempts to “correct” sample selection for systematic bias, is guesswork. There is no accepted general mathematical or scientific theory for correcting samples for systematic bias, let alone in such a multidimensional application as politics. It’s all “heuristics”—a fancy word for guesswork.

3. Underestimating Voters’ Intelligence.The late misanthropic pundit H.L. Mencken once said, "No one ever went broke underestimating the intelligence of the American public.” While that might be a good rule for predicting the audience for the 167th sequel of Spiderman, is it a reliable rule for political polling? Is there really no floor under the public’s voting intelligence?

It seems enough that political pundits assumed, from a mere half-century of data, that deep losses for the president’s party in midterm elections are a law of life. But what about assuming, as an equally universal truth, that voters blame the president for whatever bad thing is happening at the moment? Might not that be a step too far? Is there literally no limit to the average voter’s stupidity?

Although sometimes I couldn’t help myself, I tried not to read polls or prognostications less than a month before the elections. But here’s a doozy that I couldn’t help reading because all our media repeated it endlessly and made it impossible to avoid:

“Inflation is high. Voters are worried and angry. The Republicans are blaming it all on the Democrats, and President Biden has low approval ratings. Therefore voters will vote Republican.”

Any freshman student in logic could poke multiple holes in that miserable excuse for a syllogism. The most glaring omissions are two: (1) the absence of any reason to believe the Republicans, and (2) the absence of any credible plan to make things better.

In repeating this broken syllogism ad infinitum and ad nauseum, the media not only assumed that hypothetical voters have no floor under their intelligence. They also encouraged voters not to think and, in the process, used their undeniable power and influence to propagate Republican propaganda.

This is journalism?

I could go on and on, but I think I’ve made the basic point. The kinds of statistics that pollsters claim to use to predict elections were, by and large, developed to predict the behavior of atoms and molecules in numbers like Avogadro’s number, 6.02 x 10²³, which reflects the near-infinite quantity of atoms or molecules in human-sized volumes. Yet for purposes of thermodynamics and statistical mechanics, atoms and molecules have only a few relevant variables: their masses, their electrical charge (if any), and their velocities in three dimensions. Do you think that human beings, each of whom has three billion base pairs in his or her DNA, might be a bit more complicated and subtle?

Statistical mechanics works in physics because the number of variables (“degrees of freedom”) for each particle is usually less than ten, and the number of particles—6 with 23 zeroes after it—is incomparably larger, so individual differences don’t matter. The same techniques don’t work reliably with people because the number of degrees of freedom are many orders of magnitude (powers of ten) larger and the relative number of objects even smaller.

Pollsters try to account for these huge differences by varying the sample size and selection, but there is no reliable mathematical way to vary them, or even to figure out how to vary them. The answer invariably depends on the variability and plasticity of the human psyche, as in the case of abortion mentioned above. So the very application of the same mathematical methods used for molecular gasses to people is based on a fundamental misconception.

Sure, the larger the sample size, the better the prognostication. That simple relationship seems like a truism. But what if you need to “sample” just about the whole voting population to get accurate results? Isn’t that what we call an “election”?

I’m not aware of any mathematical technique, whether statistical or otherwise, that can predict how an individual human being will react to ambiguous data, whether he or she will pick up a landline or cell phone and open up to a stranger, what things will rise to the top of his or her importance list in the voting booth (or at home in filling out an absentee ballot), or whether a change of mind might occur between the polling and the election.

So there is absolutely no reliable mathematical basis for adjusting sample size, or for predicting elections among millions of voters by polling mere thousands, or even tens of thousands—let alone in cases where the polling sample is likely to be systematically skewed, and/or the electorate is closely divided, emotional issues rule, and the “hidden persuaders” are working overtime to motivate decisions based on lies, fear, name-calling and trivia.

So henceforth, for myself and my personal sanity, I intend to apply four hard rules. First and foremost, I will regard electoral polls as the modern equivalents of reading the entrails of dead birds, or the patterns of tea leaves left in a cup of tea after drinking. Second, I will hang up on any unknown caller not just after I hear the unmistakable background sounds of a “boiler room,” but even if I don’t hear a single-voiced “hello” within four seconds. Third, I will studiously avoid answering polls that try to reach me by way of any of the other five current means (with no doubt more to come) that communication technology now allows. (BTW, how accurate does anyone think Elon Musk’s recent Twitter poll was, which he cited for restoring the Demagogue’s Twitter access?) Finally, I will renew my registration on the FTC’s “Do Not Call” list, although I have little faith that our broken Congress and dispirited regulators will ever restore its long-degraded efficacy.

As for our media, they ought to leave lies, disinformation, misinformation and propaganda to the partisan political “operatives” who are already far too numerous and clever for our species’ good. They would improve their public impact immensely by reporting what (if any) practical solutions to real problems candidates for office propose. Rather than speculate endlessly on future results, reporters could analyze whether the solutions that each candidate proposes make sense, or at least telephone a few deep thinkers to do that analysis for them.

But they ought to leave the “horse race” to the handicappers of actual horse races. At least they have at their fingertips—in exhaustive statistics of horses’ past performance—something far more accurate, reliable, relevant and theoretically sound than political polls based on the unwarranted assumption that samples are properly “corrected“ for bias in multiple unknown and usually unconsidered dimensions.


For brief descriptions of and links to recent posts, click here. For an inverse-chronological list with links to all posts after January 23, 2017, click here. For a subject-matter index to posts before that date, click here.

Permalink to this post