A Polling Primer

All is not fair in love and politics.

Polls are snapshots of a river in motion. If the current is slow, the poll’s accuracy may have some longevity. If the river is flowing fast, the snapshot is old news almost immediately. So when things change rapidly, polls are more suited to being sepia-toned reminders in a scrap book than as predictors of future behavior.

That’s why I’m not relying on the polls released immediately following the Republican love-fest in Milwaukee that anointed Donald Trump, or the polls that followed the Joe Biden handoff to Kamala Harris. The river is flowing too fast for them to be accurate today.

A little Inside Baseball here. I promise not to get too deep in the weeds. I spent a lot of years fielding and analyzing consumer research. In the long past days of landline telephones, we caught people at home, and asked our questions and got out fast, so as not to be a huge bother, considering the dinner hour is the researcher’s Golden Hour.

Then people got fed up with the ballooning numbers of uninvited and disruptive phone calls, and respondents got harder to find. Next, mobile phones took over. Initially, those numbers were unlisted, then savvy consumers blocked unknown callers, and the universe of respondents shrunk dramatically. Statistically valid samples got harder and harder to produce.

Let the buyer beware.

The ongoing problem that polling companies and their university and media partners have is something known in the trade as “qualified refusals.” That term of art refers to people they’d like to include in their polling, either due to demographics, socioeconomics, or geography, but who have no interest in talking to pollsters. To boost their sample sizes to a level of statistical reliability, they have to extrapolate from the information they have gathered. Some, as a result, this kind of “quantitative” research—polling that uses large sample sizes to predict behavior, a dubious prospect to begin with—became less reliable.

You’ll notice now that research companies, via their academic and media partners, talk a lot about focus groups, a type of “qualitative” research that distinguishes itself from the big quantitative studies by gathering a small group (usually less than a dozen), and starting with broad questions, moderators drill down to specifics. Qualitative research yields opinions, but the small samples are not statistically valid for the purposes of prediction. (And yes, my fellow nerds, this is a wildly simplified explanation, but give me a break, we’re not doing open heart surgery here.)

One of my earliest colleagues in the field of music research (we asked audience members to pick the hits) saw an opportunity to expand his business and started what is now the biggest exit poll company. It went well at first, meaning that it was very accurate. Then a curious thing happened: even exit polling lost qualified respondents, and therefore, credibility.

About two thirds of voters in 2020 didn’t actually go to a polling place; they voted by mail in one form or another of absentee balloting. That reduced the statistical reliability of exit polls by, drum roll, please: two thirds. To make up for the loss of respondents, pollsters had to resort to the methodologies we questioned above, and the results were predictably skewed. Again, the telephone proved about as useful as smoke signals, and we were right back where we started. People like voting by mail, so it’s reasonable to expect the samples in exit polling to be light again.

But, the ultimate exit poll is the counted vote, and while it does not have exit polling’s immediacy, it’s not far behind in terms of time. Instead of having an accurate look at outcomes as enough people leave the booth, we’re going to have to wait a few hours, or perhaps days.

Finally, and speaking of letting the buyer beware, I wish to caution you against the innocent-seeming “push poll.” What is a push poll? It’s the wolf in sheep’s clothing of political marketing. The push poll appears unbiased and to honestly seek the respondent’s opinion, but it’s about as innocent as a “model” on Only Fans. It’s true purpose is to lull the respondent into a sense of false security.

According to Wikipedia, “A push poll is an interactive marketing technique, most commonly employed during political campaigning, in which a person or organization attempts to manipulate or alter prospective voters' views under the guise of conducting an opinion poll. Large numbers of voters are contacted with little effort made to collect and analyze their response data. Instead, the push poll is a form of telemarketing-based propaganda and rumor-mongering masquerading as an opinion poll. Push polls may rely on innuendo, or information gleaned from opposition research on the political opponent of the interests behind the poll.

We’re wise to take most polling with a grain of salt.

©2024 Jon Sinton