Lies, damned lies, and political polls.

Science is real. This time shoot the messenger, not literally.

A couple of weeks ago, after NCEA results came out, my son’s enrolment at Auckland Uni for this year was confirmed - he is doing a BSc majoring in Statistics. Well that is the plan now, who knows what will take his interest once he starts.

I spent a bit of time helping my clever young man look at the different options available and explaining what things like paper restrictions, prerequisites, and co-requisites meant. It felt very familiar, I did a BSc at Auckland Uni too - almost 30 years ago. My major was in Computer Science, although I did a couple of years of stats.

As far as ivory towers go there is nothing aesthetically pleasing about the Maths and Physics building that houses Computer Science and Statistics.

Beyond nostalgia I thought I’d write a bit about statistics following the political polls this week. Seeing the online comments it seemed to me the lack of even a basic understanding of statistics by those commenting was quite depressing.

There are aspects of polling that should be questioned, but there are also elements that are quite valid which people questioned. We’ve become attuned to questioning everything the media tells us, often for good reason.

But shrieking “Fake News” at aspects that are in fact sound is not where we want to go. That way lies Trump supporters who end up believing alternate facts, or anti vaxxers that think scientists are conspiring against them in an evil plot.

I’d like to cover three topics:

• The polling methodology

• What is polled and when

• How results are presented

The polling methodology

So in Stats 101, or hopefully before that at school, we’re taught about sample sizes and how we can gain information about a larger group, or population, by analysing a subset, with a degree of confidence - a margin of error. The larger the sample size the smaller the margin of error.

The margin of error tells you how many percentage points your results will differ from the real population value.

Most political polls are of approximately 1,000 people. A smaller sample is too inaccurate and a much larger sample is not worth the additional cost. What is important is how we select who is in our sample.

Say for example we wanted to assess the proportion of people in Aotearoa who like Hip Hop music. Now it would be impractical to ask everyone so we’d look at a random sample from which we can ascertain information about the wider population. When we poll our 1,000 people, we need to be careful that they are a true random sample. If we were to ask people outside a Snoop Dogg concert we might get quite a different result than at the Gore cosmopolitan club.

In the past polling organisations relied on the fact that the vast majority of people had landline phones. If you selected random numbers from phone books you got a reasonably random sample. So that was good. But now a lot of people don’t have landlines and those who do are skewed heavily towards a particular demographic.

So the companies do the best they can with mobiles and with people off the internet. A self selecting population badly impacts the accuracy of a poll. If you’re reliant on people having registered online to take part those stats aren’t really worth much - about as much as a Newshub survey.

People complain about a couple of things around polling - “the sample size is too small”, or more frequently “how come I’ve never been polled then?”

I saw someone suggest this week that they would have more confidence in the result if the poll had been of 250,000 people rather than 1,000. Which could be a tad impractical and actually doesn’t increase accuracy significantly.

The size of the sample is not the issue. Having an opinion on things like sample sizes is as valid as having an uninformed opinion on the effectiveness of the Covid vaccinations or whether gravity does what some people believe it does.

“But, I’ve never been polled” - well cool. I’ve never been to the moon, but I don’t doubt that others have been.

The estimated number of people in Aotearoa who are 18 or over is 3,989,000. Now last year there were nine polls undertaken between our two News organisations. If you include all the other polls, Roy Morgan, Taxpayer’s Union, internal polls, etc, there were 41 in total.

About 40,000 people, or slightly less as some may have been polled more than once. The equivalent of a town the size of Whanganui. So saying you’ve never been polled is kind of like walking around mystified saying “well, I don’t live in Whanganui!”

The bridge to Whanganui.

Of course there are another 40,000 people each subsequent year, but even after ten years we’re still only talking the population of Christchurch. It would take a hundred years for everyone to get polled an average of once. If it helps I’ve been polled twice.

Before we move on let’s take a look at those two polls announced on Monday, which had very similar results.