Track Record: 2010 Errors

This page first posted 16 May 2010

• The Conservatives will win the most votes and the most seats
• Labour and the Lib Dems are fighting for second and third place, but Labour will get more seats
• Parliament will probably be hung.

In numerical terms, the prediction and the outcome were:

The Conservatives were predicted relatively accurately, if slightly low. The main error was that Labour was under-estimated and the Liberal Democrats were over-estimated. This error was driven by the over-estimation of Lib Dem percentage support by both the pollsters and the spread betting markets. On a seat-by-seat basis, 80 seats were mis-predicted in total.

We will now look at these and other issues in more detail. The particular topics studied are:

1. Opinion poll error
2. Regional swing effects
3. Idiosyncracies

1. Opinion poll error

Ipsos-MORI was clearly the most accurate of the pollsters at this election, and ICM was the next best. The figures from the spread betting market were not perfect, but were more accurate than most of the pollsters. Adding the spread betting numbers in our average helped it a little, but in the end it was not a very important factor.

The overall accuracy of the pollsters was good at this election. The most important number to watch is the estimated lead of the Conservatives over Labour (or vice versa), which the pollsters had at 7.9% and which turned out to be 7.3%. This is a marked improvement to previous elections (during 1992-2005) when the pollsters consistently over-estimated Labour's support at the expense of the Conservatives.

The main error this year was the over-estimatation of the Liberal Democrats' vote share. This error was common to both the pollsters and the spread betting markets. The pollsters average error for this was 3.7%. This is equivalent to twelve standard deviations (assuming uniform unbiased polling), which suggests that there is some underlying problem in their methodology.

If the pollsters had been 100% accurate, then the Electoral Calculus prediction would have been: Con 299, Lab 255, LibDem 65, and only sixty-three seats would have been mis-predicted. This would have been a good prediction, which has all three major parties within 8 seats of the correct answer.

In summary, this was a pretty good performance by the pollsters, apart from the over-estimation of the Liberal Democrats. This polling error is the main cause of our error in predicting the Lib Dems seats.

2. Regional swing

There were three separate polls conducted. The sample dates were 4-11 April, 11-18 April and 18-25 April. We used all three polls but applied a relative weighting of 25%, 50% and 100% respectively to give more weight to the recent polls.

The accuracy of the YouGov regional polls is patchy. Only about half of the swings have the correct sign. Looking at individual regions, Yorks/Humber is good, as is the West Midlands. But the North West, South West, London and South East are not so good. London particularly was a problem. YouGov thought that the Conservatives were doing relatively strongly there, when in fact they did poorly and Labour did very well.

Despite this error, it had little effect on the prediction's accuracy. Even if the regional polling had been perfectly accurate, the prediction would have changed little. It would have been: Con 299, Lab 254, LibDem 66, and fifty-five seats would have been mis-predicted. This is close to the UNS-based result in section one, though eight fewer seats are mis-predicted (several in London and the North West).

3. Idiosyncracies

For instance, the Conservatives gained 25 seats and lost 17 seats unexpectedly, Labour gained 20 and lost 16, and the LibDems gained 9 and lost 18.

We can look at the actual seats concerned, to see whether there is any pattern, or if it is just random noise.

[Note the Slide-O-Meter notation of "CON-03" to mean a Conservative majority of 3% is used in this table. Majorities are rounded to the nearest integer percentage, so "CON-00" means a majority of less than 0.5%.]

One story here is incumbency. A seat where the incumbent MP is re-elected is marked with an asterisk (*). Where a party already holds the seat, it proved difficult for the Conservatives to dislodge. Popular incumbent MPs on this list include Gisela Stuart (Birmingham Edgbaston), Chris Huhne (Eastleigh) and David Heath (Somerton and Frome).

The Liberal Democrats did well to pick up Eastbourne and Wells, and Labour did well to retain Sefton Central and Barrow & Furness without incumbency. Luton South (Margaret Moran's old seat) was surprisingly held by Labour, perhaps because LibDems and independents (such as Esther Rantzen) split the anti-Labour vote.

Now let us turn attention to the sixteen seats which Labour was predicted to win, but did not.

Incumbency was not a positive factor here, because Labour lost all these seats which it used to hold. Incumbency could have been a negative factor in some of these seats, where the previous MP had suffered from the expenses scandal of 2009. Those include Tony McNulty (Harrow East), Ann Keen (Brentford and Isleworth) and Kitty Ussher (Burnley). The one positive incumbent was Sarah Teather (Lib Dem, Brent Central) whose neighbouring seat was removed by boundary changes and stood successfully in this new seat.

Norwich North and Crewe & Nantwich were by-election gains by the Conservatives during the last parliament, which they retained. Three seats were marginal, and could have gone either way.

That leaves seven seats which Labour unexpectedly lost (six to the Conservatives, and Redcar to the Liberal Democrats). There is no obvious pattern to those seats, except they are mostly non-urban.

Thirdly, let us look at the eighteen Liberal Democrats surprising seats:

Four seats were retained by incumbents, or five if we count Glenda Jackson in Hampstead and Kilburn. In five other seats, the sitting Lib Dem MP stood down at this election and the party's new candidate was not elected. This is the flip-side of incumbency - if the vote is a personal vote for the old MP then it does not transfer over in full to the party's new candidate. York Outer was similar in that it was a new nominal Lib Dem seat, but there was no sitting MP.

Of the other seven seats, Rochdale was a surprise win for the Lib Dems in 2005, helped by an anti-war student vote which has presumably now unwound. Richmond Park saw the incumbent Susan Kramer defeated by high-profile Zac Goldsmith (Con). The other five seats were focused in the south west, but with no obvious pattern.

Finally, there were four other mis-predicted seats:

Three independents lost their seats, and Ynys Mon (Anglesey) was held by the incumbent Albert Owen MP.

Summary and Conclusions

• Our headline prediction for 2010 was fairly accurate
• The main systematic error was the over-estimation of the Liberal Democrats, caused by inaccurating polling
• Regional effects were present but not very significant at this election
• There was a large amount of idiosyncratic seat mis-predictions, caused by several factors:
  • Incumbency, or the lack of it
  • The parliamentary expenses scandal
  • By-election gains in the previous parliament
  • Other local / unidentified factors