However, the survey will with luck go down as a classic example of how not to do a survey. It exhibits all the faults that Darell Huff described in his wondrous, if now dated, book "How to lie with statistics"
What was wrong with it?
The premise was simple: email all the members and tot up those who responded.
The problem with that it is not random. Now the number of respondees is reasonable - 788 but that number has what is known as a "selection bias".
For instance:
- The survey was performed on-line. In other words there was a selection here: only those people who has access to the Internet could respond. Those who didn't couldn't. People who have access to the Internet might have different views to those who don't. (If anyone thinks that this is unlikely, this actually happened in a more extreme for in the 1948 US Presidential election when a telephone poll gave Dewey a big win over Truman. The problem was that the people who owned telephones in 1948 were then wealthier than those who didn't, and hence less likely to vote for the Democrat incumbent).
- Secondly, the 788 respondees were only those people who could be arsed to email something back. Now these people almost by definition are more highly motivated that those who didn't. How this translates into a more accurate poll answer is literally impossible to say - which is exactly the point.
- A poll must be be random across the complete sample that is being measure - ie all the grassroots members and must be of a size that makes it results significant (both in the statistical and political sense)
The combination of the three completely invalidates the poll - making it one that Sir Robert Worcester of MORI fame calls a "Voodoo Poll" and it makes all the worse that a paper such as the Independent could splash it across its front pages this morning. It literally is meaningless
No comments:
Post a Comment