No. of Recommendations: 4
A new national NPR/PBS NewsHour/Marist poll finds "Biden actually gained a point since last month’s survey, which was taken before the debate. In this poll, he leads Trump 50% to 48% in a head-to-head matchup. But Biden slips when third-party options are introduced, with Trump holding the slightest advantage with 43% to 42%.
“Gained a point” is meaningless in a poll which has a 3.6% margin of error. That’s because the margin of ERROR is real, and a 42% number is statistically identical to a 45% number. In a poll of 1,200 or so people that’s typical, even if every other protocol has been scrupulously followed : it’s not just cell phones, it’s not just home phones, it’s geographically spread through the country, if care is taken to hit minority *and* wealthy enclaves (hard to get responses from either), if young people are surveyed accurately (also hard to reach) and if it’s a stratified, randomized sample and doesn’t contain any language loading, or even positional bias (first questions get answered more often than those further down the list) and so on.
But wait! It’s even worse than that. Usually unsaid is there is still something called “confidence level”, which means that even if every single thing that can be controlled for is done right, the survey will still be wrong 1 out of 10 times if the confidence level is only 90%. (It’s generally higher, maybe 93-95%).
Let me offer a perfectly perfect poll, where the candidate gets 50%, exactly, and it’s true.
But the polls he gets are 50%. A week later 49%. A week after that 48%. And a week after that 45%. Remember, this candidate has 50% and that’s perfectly accurate, but the polls going down to 49% and 48% are within the margin of error, and the last poll, at 45% is that weird time when the confidence factor kicked in (or out) and the poll is just dead wrong. Eventually that will wash out and self correct, however by that time the candidate has slit his wrists.
In my professional life I watched this play out; radio stations were “surveyed” every month, and stations with rock solid listening (like, say so-called “Beautiful Music” stations) would jump up and down for no apparent reason except “sample wobble”, and would suddenly jump back to where they had been - all without explanation. (Some stations, like all news or sports can vary widely based on the programming, but that is a different thing.)
Having a plethora of surveys is good, but it’s the *trend* of each individually that matters, as they all have their own inherent surveying techniques both for good and bad, so quoting a single survey is fraught. That’s what makes political polling so dicey: short season (even if it seems interminably long to us) and random events that spike.
Anyway, polls are media crutches, but the public loves them so they’re not going away. Just every once in a while I feel compelled to shout at the internet and say “Don’t pay so much attention”, but it does no good, and I accept that.
Ah. There. I feel better now.
