Opinion Polling Science

This page first posted 31 July 2004

Professional Opinion Pollsters strive to get accurate results, but there are still various sources of error which have to be monitored or controlled.

Statistical Sampling Error

The sample of voters polled is meant to be representative, but it is randomly selected. This randomness gives rise to possible error, because it is possible that we are unlucky and happen to over-sample from supporters of a particular party.

The margin of error of an opinion poll is often quoted as plus or minus 3%. This is a good approximation, but there is an exact formula that can also be used.

If an opinion poll sampled $N$ people, and found a proportion $p$ of support for a party, then the margin of error on that proportion is $$ Err = 1.96\sqrt{p(1-p)\over N}. $$ For instance if a poll of $N=5,000$ people estimates a party at $p=30\%$ support, then the margin of error is $$ Err =1.96\sqrt{{0.3\times 0.7}\over 5000}=1.3\%. $$ Another example is a poll of 1000 people, and a party at 50% support, where the error is 3.1%, which is the famous "3%" margin.

This margin is not an absolute guarantee but a confidence limit. It should be expected that for 19 polls out of 20 the true levels will lie within the margin of error (95% confidence level). The scaling constant of 1.96 is the inverse of the normal distribution function at 97.5% (for a two-sided confidence interval of 95%).

Representativeness and Quotas

The aim is for the sample to fairly represent each of every sort of voter. But there are many practical difficulties. A busy businessman might refuse an invitation to a face-to-face interview (giving a pro-Labour bias to the poll), while internet polling will exclude those without access to computers (possibly causing an anti-Labour bias). Others may refuse to reveal their intentions, or provide misleading answers.

Pollsters try to cope with these problems by various means, including quota sampling. The sample is weighed to match some key national averages, such as the proportion of people who are: owner-occupiers, council tenants, members of trade unions, in a particular social class, and so on.

Quota-sampling aims to reduce variance, but in can introduce biases of its own, and was blamed in part for the polls' failure in 1992. The track record of the polls since 1992 has been better, but not perfect.

© Electoral Calculus.