Track Record: 2010 Errors

This page first posted 16 May 2010

Our headline prediction for the May 2010 election was that (in the subscriber email dated 5 May 2010): This scores about two and a half out of three for accuracy. In fact Labour clearly got more votes than the Lib Dems, but the other predictions were accurate.

In numerical terms, the prediction and the outcome were:

Party2005 Votes2005 SeatsPred VotesPred Seats Actual VotesActual Seats Vote ErrorSeat Error
NAT 2.22%8 2.22%112.26%9+0.04%-2
MIN 5.71%21 7.18%217.54%19+0.36%-2

The Conservatives were predicted relatively accurately, if slightly low. The main error was that Labour was under-estimated and the Liberal Democrats were over-estimated. This error was driven by the over-estimation of Lib Dem percentage support by both the pollsters and the spread betting markets. On a seat-by-seat basis, 80 seats were mis-predicted in total.

We will now look at these and other issues in more detail. The particular topics studied are:

  1. Opinion poll error
  2. Regional swing effects
  3. Idiosyncracies
There is a separate analysis of model success.

1. Opinion poll error

To make our prediction, we used an average of the final campaign polls taken by recognised polling organisations (members of the British Polling Council), along with implied support figures taken from the spread betting markets (Sporting Index).

PollsterSample datesSample sizeCON%LAB%LIB%Error%
News of the World/Ipsos-MORI23 Apr 10 - 23 Apr 101,2453630231.9%
The Guardian/ICM3 May 10 - 4 May 102,0223628265.1%
The Sun/YouGov4 May 10 - 5 May 106,4833528288.1%
The Times/Populus4 May 10 - 5 May 102,5053728275.1%
Political Betting/Angus Reid4 May 10 - 5 May 102,28336242912.1%
Daily Mail/Harris4 May 10 - 5 May 103,4063529276.1%
The Independent; ITV/ComRes4 May 10 - 5 May 101,0253728286.1%
POLL AVERAGE23 Apr 10 - 5 May 1018,96935.727.827.36.9%
Sporting Index (implied)5 May 10 - 5 May 1010,00036.827.226.35.4%
OVERALL AVERAGE23 Apr 10 - 5 May 1028,96936.127.626.96.3%
ELECTION 20106 May 201037.029.723.60.0%

Ipsos-MORI was clearly the most accurate of the pollsters at this election, and ICM was the next best. The figures from the spread betting market were not perfect, but were more accurate than most of the pollsters. Adding the spread betting numbers in our average helped it a little, but in the end it was not a very important factor.

The overall accuracy of the pollsters was good at this election. The most important number to watch is the estimated lead of the Conservatives over Labour (or vice versa), which the pollsters had at 7.9% and which turned out to be 7.3%. This is a marked improvement to previous elections (during 1992-2005) when the pollsters consistently over-estimated Labour's support at the expense of the Conservatives.

The main error this year was the over-estimatation of the Liberal Democrats' vote share. This error was common to both the pollsters and the spread betting markets. The pollsters average error for this was 3.7%. This is equivalent to twelve standard deviations (assuming uniform unbiased polling), which suggests that there is some underlying problem in their methodology.

If the pollsters had been 100% accurate, then the Electoral Calculus prediction would have been: Con 299, Lab 255, LibDem 65, and only sixty-three seats would have been mis-predicted. This would have been a good prediction, which has all three major parties within 8 seats of the correct answer.

In summary, this was a pretty good performance by the pollsters, apart from the over-estimation of the Liberal Democrats. This polling error is the main cause of our error in predicting the Lib Dems seats.

2. Regional swing

For the 2010 election campaign we did use a regional swing model, to improve on the simplisitic Uniform National Swing model. The details of this are described in the article on 2010 campaign methodology. The basic idea is to get estimates of how regions differ from the overall national picture, and apply those fixed offsets to the changing national situation. This needs large-scale regional polling in order to calculate the fixed offsets (called differential regional swings) in each region. To do this, we used three large-scale polls by YouGov for Politics Home. The details are in a YouGov pdf file (link now broken).

There were three separate polls conducted. The sample dates were 4-11 April, 11-18 April and 18-25 April. We used all three polls but applied a relative weighting of 25%, 50% and 100% respectively to give more weight to the recent polls.

YouGov Differential SwingsActual Differential Swings
AreaCon %Lab %Lib %        Con %Lab %Lib %
The North-0.2-
North West0.5-1.40.3-0.61.2-0.8
West Midlands2.3-
East Midlands-0.8-
South West-
South East0.21.3-2.01.2-1.60.2

The accuracy of the YouGov regional polls is patchy. Only about half of the swings have the correct sign. Looking at individual regions, Yorks/Humber is good, as is the West Midlands. But the North West, South West, London and South East are not so good. London particularly was a problem. YouGov thought that the Conservatives were doing relatively strongly there, when in fact they did poorly and Labour did very well.

Despite this error, it had little effect on the prediction's accuracy. Even if the regional polling had been perfectly accurate, the prediction would have changed little. It would have been: Con 299, Lab 254, LibDem 66, and fifty-five seats would have been mis-predicted. This is close to the UNS-based result in section one, though eight fewer seats are mis-predicted (several in London and the North West).

3. Idiosyncracies

After allowing for polling error both nationally and regionally, we are left with a prediction which is quite good for total numbers of seats (Con 299, Lab 254, LibDem 66), but has fifty-five seats mis-predicted. This is relatively high, but had an unusually small effect on the total outcome. Parties were winning and losing seats unexpectedly, but without any definite trend.

For instance, the Conservatives gained 25 seats and lost 17 seats unexpectedly, Labour gained 20 and lost 16, and the LibDems gained 9 and lost 18.

We can look at the actual seats concerned, to see whether there is any pattern, or if it is just random noise.

Predicted Conservative seats which were not
NumSeatGE 2005PredictionGE 2010County (Area)Comment
1Luton SouthLAB-15CON-00LAB-06Bedfordshire (Anglia)Urban
2SolihullLIB-01CON-00LIB-00West Midlands (West Midlands)Marginal (*)
3Somerton and FromeLIB-00CON-00LIB-03Somerset (South West)South West (*)
4Barrow and FurnessLAB-12CON-01LAB-12Cumbria (The North)NW semi-urban
5Brighton PavilionLAB-13CON-01MIN-02East Sussex (South East)Greens, marginal
6Dudley NorthLAB-11CON-01LAB-02West Midlands (West Midlands)Marginal (*)
7Taunton DeaneLIB-00CON-01LIB-07Somerset (South West)South West (*)
8TynemouthLAB-12CON-01LAB-11Newcastle area (The North)Urban (*)
9EastleighLIB-01CON-02LIB-07Hampshire (South East)South town (*)
10CopelandLAB-10CON-03LAB-09Cumbria (The North)NW rural (*)
11EastbourneCON-02CON-03LIB-07East Sussex (South East)South town
12Sefton CentralLAB-06CON-03LAB-08Merseyside (North West)Urban
13GedlingLAB-09CON-04LAB-04Nottinghamshire (East Midlands)Urban (*)
14HalifaxLAB-09CON-05LAB-03West Yorkshire (Yorks/Humber)Urban (*)
15WellsCON-06CON-07LIB-01Somerset (South West)South West
16Birmingham EdgbastonLAB-04CON-08LAB-03Birmingham (West Midlands)Urban (*)
17Bradford WestLAB-06CON-08LAB-14West Yorkshire (Yorks/Humber)Urban (*)

[Note the Slide-O-Meter notation of "CON-03" to mean a Conservative majority of 3% is used in this table. Majorities are rounded to the nearest integer percentage, so "CON-00" means a majority of less than 0.5%.]

One story here is incumbency. A seat where the incumbent MP is re-elected is marked with an asterisk (*). Where a party already holds the seat, it proved difficult for the Conservatives to dislodge. Popular incumbent MPs on this list include Gisela Stuart (Birmingham Edgbaston), Chris Huhne (Eastleigh) and David Heath (Somerton and Frome).

The Liberal Democrats did well to pick up Eastbourne and Wells, and Labour did well to retain Sefton Central and Barrow & Furness without incumbency. Luton South (Margaret Moran's old seat) was surprisingly held by Labour, perhaps because LibDems and independents (such as Esther Rantzen) split the anti-Labour vote.

Now let us turn attention to the sixteen seats which Labour was predicted to win, but did not.

Predicted Labour seats which were not
NumSeatGE 2005PredictionGE 2010County (Area)Comment
18KingswoodLAB-12LAB-00CON-05Bristol area (South West)
19Norwich NorthLAB-15LAB-00CON-09Norfolk (Anglia)By-election gain
20Warwickshire NorthLAB-14LAB-00CON-00Warwickshire (East Midlands)Marginal
21Weaver ValeLAB-13LAB-00CON-02Cheshire (West Midlands)Marginal
22Harrow EastLAB-07LAB-01CON-07Harrow (London)Expenses
23Brentford and IsleworthLAB-08LAB-02CON-04Hounslow (London)Expenses
24ErewashLAB-17LAB-02CON-05Derbyshire (East Midlands)Urban
25HendonLAB-08LAB-02CON-00Barnet (London)Marginal
26Crewe and NantwichLAB-16LAB-03CON-12Cheshire (West Midlands)By-election gain
27Morecambe and LunesdaleLAB-12LAB-03CON-02Lancashire (North West)Semi-urban
28SherwoodLAB-20LAB-06CON-00Nottinghamshire (East Midlands)Semi-urban
29Cannock ChaseLAB-21LAB-07CON-07Staffordshire (West Midlands)Semi-urban
30Lancaster and FleetwoodLAB-17LAB-08CON-01Lancashire (North West)Semi-urban
31BurnleyLAB-15LAB-09LIB-04Lancashire (North West)Expenses
32Brent CentralLAB-22LAB-19LIB-03Brent (London)London
33RedcarLAB-31LAB-20LIB-12Cleveland (The North)Urban

Incumbency was not a positive factor here, because Labour lost all these seats which it used to hold. Incumbency could have been a negative factor in some of these seats, where the previous MP had suffered from the expenses scandal of 2009. Those include Tony McNulty (Harrow East), Ann Keen (Brentford and Isleworth) and Kitty Ussher (Burnley). The one positive incumbent was Sarah Teather (Lib Dem, Brent Central) whose neighbouring seat was removed by boundary changes and stood successfully in this new seat.

Norwich North and Crewe & Nantwich were by-election gains by the Conservatives during the last parliament, which they retained. Three seats were marginal, and could have gone either way.

That leaves seven seats which Labour unexpectedly lost (six to the Conservatives, and Redcar to the Liberal Democrats). There is no obvious pattern to those seats, except they are mostly non-urban.

Thirdly, let us look at the eighteen Liberal Democrats surprising seats:

Predicted Liberal Democrat seats which were not
NumSeatGE 2005PredictionGE 2010County (Area)Comment
34Islington South and FinsburyLAB-02LIB-00LAB-08Islington (London)London (*)
35Durham, City ofLAB-07LIB-02LAB-07Durham (The North)Urban (*)
36Hereford and South HerefordshireLIB-03LIB-02CON-05Hereford and Worcs (West Midlands)Retiring MP
37Leicester SouthLAB-09LIB-02LAB-19Leicestershire (East Midlands)Urban (*)
38Hampstead and KilburnLIB-01LIB-03LAB-00Camden (London)Nominal only
39York OuterLIB-05LIB-03CON-07North Yorkshire (Yorks/Humber)Nominal only
40Richmond ParkLIB-07LIB-05CON-07Richmond upon Thames (London)London
41Cornwall South EastLIB-08LIB-06CON-06Cornwall (South West)Retiring MP
42RochdaleLIB-01LIB-06LAB-02Eastern Manchester (North West)Anti-war unwind
43Oxford EastLAB-01LIB-07LAB-09Oxfordshire (South West)Urban (*)
44Truro and FalmouthLIB-10LIB-08CON-01Cornwall (South West)Retiring MP
45Camborne and RedruthLIB-07LIB-09CON-00Cornwall (South West)Big swing
46Newton AbbotLIB-14LIB-12CON-01Devon (South West)Big swing
47Oxford West and AbingdonLIB-14LIB-12CON-00Oxfordshire (South West)Big swing
48WinchesterLIB-15LIB-12CON-05Hampshire (South East)Retiring MP
49ChesterfieldLIB-07LIB-17LAB-01Derbyshire (East Midlands)Urban
50Harrogate and KnaresboroughLIB-19LIB-19CON-02North Yorkshire (Yorks/Humber)Retiring MP
51MontgomeryshireLIB-23LIB-20CON-04Powys (Wales)Rural

Four seats were retained by incumbents, or five if we count Glenda Jackson in Hampstead and Kilburn. In five other seats, the sitting Lib Dem MP stood down at this election and the party's new candidate was not elected. This is the flip-side of incumbency - if the vote is a personal vote for the old MP then it does not transfer over in full to the party's new candidate. York Outer was similar in that it was a new nominal Lib Dem seat, but there was no sitting MP.

Of the other seven seats, Rochdale was a surprise win for the Lib Dems in 2005, helped by an anti-war student vote which has presumably now unwound. Richmond Park saw the incumbent Susan Kramer defeated by high-profile Zac Goldsmith (Con). The other five seats were focused in the south west, but with no obvious pattern.

Finally, there were four other mis-predicted seats:

Predicted Other Party seats which were not
NumSeatGE 2005PredictionGE 2010County (Area)Comment
52Ynys MonLAB-03NAT-02LAB-07Gwynedd (Wales)Incumbent (*)
53Wyre ForestMIN-09MIN-04CON-05Hereford and Worcs (West Midlands)Dr Richard Taylor
54Bethnal Green and BowMIN-05MIN-07LAB-23Tower Hamlets (London)Respect
55Blaenau GwentMIN-26MIN-32LAB-32Mid Glamorgan and Gwent (Wales)Independent Labour

Three independents lost their seats, and Ynys Mon (Anglesey) was held by the incumbent Albert Owen MP.

Summary and Conclusions

The main points that this analysis has shown are: The overall performance of the prediction was quite good, and compares well to recent years.
Return to top of page, or track record summary, or home page.