Track Record: 2015 Errors

This page first posted 16 May 2015

The headline prediction for the May 2015 election was not accurate. The final prediction was for a hung parliament with Labour/SNP as the largest bloc. The actual result was a small Conservative majority.

In numerical terms, the prediction and the outcome were:

The Conservative support was significantly underestimated, which caused the number of Conservative seats also to be underestimated. Although the Labour support figure was quite accurate, the error in the Conservatives caused the predicted number of Labour seats to be too high. The Liberal Democrats were also partly overestimated, but the prediction was accurate in saying that they would lose the vast majority of their seats.

The predictions for the other parties were relatively good, getting Plaid Cymru, UKIP and the Greens exactly right (including the exact seats held and gained), and being fairly accurate in predicting the landslide win for the SNP in Scotland.

In total sixty-four seats were mis-predicted. This would be a moderately good result if the general trend had been right, but the overall prediction quality was poor at this election. This was mostly due to polling error in the pre-election opinion polls.

We will now look at these and other issues in more detail. The particular topics studied are:

1. Opinion poll error
2. Model error
3. Idiosyncracies

1. Opinion poll error

None of the pollsters had a very good election. Every one had Conservative and Labour within one per cent of each other, compared with the actual gap of more than six per cent. At the time of writing, there is an investigation under way by the British Polling Council (the pollsters' trade body) into the polling errors. Amongst the pollsters, ComRes was the least inaccurate by a short head.

Electoral Calculus also used spread betting market prices from Sporting Index as well, because these had been successful in 2010. At this election, their performance was mixed. They were more accurate about the Conservative lead over Labour (seeing 4.3% instead of 6.6%), but they overestimated the Liberal Democrats. Their actual seat forecast was: Con 289, Lab 265, Lib 26, UKIP 3, Green 1, SNP 46. Their implied support figures were overall worse than the average of the pollsters, mostly because of their inaccurate belief in Lib Dem strength (or incumbency). Including them in the overall average made little difference in terms of vote share accuracy, but it helped in terms of seats.

2. Model error

Given the large polling errors, it is hard to tell how well the actual model performed. The model, which converts national support figures into seats, is only as good as its inputs. If the input polling data is bad, then the model output will be bad. This effect is sometimes described as "garbage in, garbage out".

But we can adjust for this by feeding the actual 2015 support levels, rather than the polling figures, into the model. Then we get the following result:

This is a much more accurate result, which helps confirm the fact that the prediction was wrong primarily because the pre-election polls were wrong. The prediction is still not quite exact. The Conservatives are a little low, and Labour are a little high, so that the Conservatives are shown just short of a majority, rather than just over. But it's not too bad, and the other parties are also predicted relatively well.

In terms of individual seats, only 36 seats are wrongly predicted, which is a good result. This compares well with 2010 when the equivalent figure was 63 mis-predicted seats, and 2005 which had 45 seats.

[Note the use of Slide-O-Meter notation of "CON-03" to mean a Conservative majority of 3% which is used in this table. Majorities are rounded to the nearest integer percentage, so "CON-00" means a majority of less than 0.5%.]

There are a number of stories here. In outline they are:

• Con strength. There were a large number (11) of marginal seats where the Conservatives just did a bit better against Labour than expected [7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 20]. It's hard to know if this was good Conservative campaigning, voter nervousness about Ed Miliband's labour party, or other factors. But it was visible across the country, particularly in the Midlands, but also further north with Morley, Bolton and Stockton falling to the Conservatives.
• Welsh Con strength. Particularly notable was the Conservative strength in Wales where they picked up three seats [5, 6, 13] against the model's expectations. Two of these, Gower [5] and Vale of Clwyd [6], were particularly striking as predicted Labour majorities of eight and nine per cent were overturned.
• Labour London strength. On a generally bad night for Labour, much of the good news for them came from the capital. They won three London seats in Enfield [25], Ealing [29] and Ilford [30] against expectations.
• Lib Dem weakness in the south west. Three seats in the south west [4, 31, 33], which the Lib Dems could have held on, fell to other parties. Overall the Lib Dem vote in the south west fall from 37% in 2010 to 17% in 2015, which was the greatest fall in any region.
• SNP strength. Overall the SNP did even better than expected and picked up three very safe Labour seats in Glasgow and Fife [1, 2, 3]. Only in Edinburgh and the borders was the SNP advance slowed much. In both Edinburgh South [26] and Dumfriesshire [32], the voting patterns indicated high-levels of tactical voting by anti-SNP voters.
• Con weakness in Merseyside. Against the general pro-Conservative trend, Merseyside was a notable exception. In two seats, Wirral West [28] and Southport [35], the Conservatives failed to win where expected.
• Anti-Tory tactical voting. This was not a big effect this year. It seems to have happened in Hove [24] near the Green's Brighton base. In Dewsbury [22], the anti-Con vote was fragmented in 2010 but it came together in 2015 for Labour to win the seat.
• Marginals. As usual there were a number of seats which were just marginal and could have gone either way. This year there were five of them [19, 21, 23, 27, 34], including the unfortunate Vince Cable in Twickenham [34].
• Strong candidates. And there were a couple of seats that just went their own way. Matthew Offord held Hendon [18] for the Conservatives despite the general anti-Conservative voting in London. And Tom Brake (Lib Dem) was re-elected in Carshalton and Wallington [36], despite the overall disastrous Lib Dem performance.

Summary and Conclusions

• Our headline prediction for 2015 was not accurate
• The primary cause of the error was poor estimation of national support levels by the pre-election polls conducted by the pollsters
• Allowing for the polling error, the model performed well with only 36 seats mis-predicted.
• The causes of seat mis-prediction were:
  • Conservative strength across the country in key seats
  • Some regional effects
  • Only a small amount of tactical voting