I never did statistics at school. Those of us doing science A-levels were made to study ‘maths with mechanics’; if you want to know how a parallelogram of forces works, or how to calculate angular momentum, I’m your man, but ask about standard deviation and margin of error and I’ll look shifty and start talking about the weather as I try to Google the answer.
But I’m numerate enough to know when figures just don’t smell right and so it is with this table doing the rounds on Twitter. It’s been posted by Andrew Hawkins, the Chairman of one of the polling companies ComRes. It tries to show how all the polling firms were within the appropriate margin of error (+/- 3 percentage points) for their final polls when you take an average of how much they were out for each party.
This is pretty disingenuous stuff (the margin of error of a survey applies at 50% for example and reduces for smaller percentages, so averaging across the big and small parties is nonsensical). Andrew could have claimed that some of the pollsters were within the margin of error (just) for the larger parties as this next table shows, but even then I think he’s on a hiding to nothing.
Sometimes things go wrong – the best laid plans screw up, the wonderful projections that were made just don’t come to pass, the great concept that everyone bought into just dies in the market. It’s not anyone’s fault, but even so you hold up your hands, admit something hasn’t worked, go away and try and learn the lessons. To do otherwise is to invite ridicule.
And so it is for the opinion polls. Something went wrong. Now that might have been to do with the sampling or the methodology, or it might have been down to people genuinely changing their minds in the final few hours. Whatever it was, the result that they had been predicting was a country mile off what actually happened. The British Polling Council has at least said that “The final opinion polls before the election were clearly not as accurate as we would like” (shades of Emperor Hirohito’s surrender speech “the war situation has developed not necessarily to Japan’s advantage“) and have announced an investigation into the causes. It will be interesting to see what they find and whether next time round anyone actually trusts what they produce.