Menu Close

Evaluating the 2019 Election Projections

Anthony Piscitelli

With the votes mostly counted and almost all seats decided, it is now possible to evaluate the election projections and predictions. For this analysis, I have used data from the CBC Canada Votes at 11:00 am on October 22, 2019.

This analysis will examine eleven poll aggregators, Four analysis-based projections, and three polling firm models. 338 Canada, Calculated Politics, Canadian Election Watch, CBC Poll Tracker, Lean Toss Up, LISPOP, Too Close to all, Visualized Politics, @EarlWashburn, @repdonsman456, and @politikstcan each aggregate multiple polls from different polling firms to predict the election. Election Prediction Project and Teddy on Politics makes predictions using an analysis process, which examines data from the public alongside general trends. Teddy on Politics and @kylejhutton made similar ‘gut-based projections’ of the election. Ekos, Forum, and Mainstreet are polling firms who use their data to model the election results.

Liberals won 157 seats. Overall, we can see that all the poll aggregators except for Lean Toss Up underestimated support for the Liberals. All the analysis-based projections also underestimated the Liberal support. However, two of three polling firms Ekos and Mainstreet overestimated support for the Liberals. Ekos was the closest to the Liberal projections missing by one seat at 158.

The Conservatives won 121 seats. Every poll aggregator projection, except for Lean Toss Up and @politicstacan, overestimated support for the Conservatives. All three polling firms underestimated Conservative support. Calculated Politics was the closest Conservative projection missing by one seat at 122.

The NDP won 24 seats. Every projection overestimated NDP support. Lean Toss Up and Mainstreet were the closest to the NDP at 25. The Bloc won 32 seats and every projection except for Ekos and Mainstreet estimated Bloc support. Calculated politics were closest to the Bloc at 34.

The Green party won three seats. Seven projections accurately projected this result. The People’s Party of Canada won zero seats. Seven projections accurately projected this result as well. One independent won, fifteen projections predicted this result.

To evaluate the overall accuracy of the different projections two methods are being used. First, the projection for each party is subtracted from the actual seats won. The absolute value is then taken (i.e. if it is negative it is turned positive) and these values are added together. The second method involves taking the Sum of Squares difference. This is done by subtracting the difference between projection and result and squaring this value then adding these together. This value is then divided by 7 (for the 7 projections) and then the square root is taken to show the average difference between each individual projection. The results of the two approaches are relatively similar.

Mainstreet was the most accurate projection by polling firm with Ekos close behind. Lean Toss Up was the most accurate poll aggregator, and third most accurate. The Elections Prediction Project was the most accurate analysis-based projection and fourth overall.

Part of the accuracy is determined as a result of the models the polling firms and aggregators use to project the election. Some of the differences in accuracy can also be attributed to how the polling aggregators model the polls of the election.

When comparing how the poll aggregators did at projecting the election Lean Toss Up once again was the most accurate only missing the actual results by an average of one percentage point. It should be noted that some of the poll aggregators did not share the projections for the PPC of Independents.

While individual models varied slightly in their performance. Overall, the aggregators, analysis-based projections, and polling firms should be satisfied with how they performed at projecting this election. All models predicted a minority government and most projected the Liberals would win the most seats.

*Updated at 1:00 pm to include the final Mainstreet seat projections

Final Federal Election Vote Projections

Anthony Piscitelli

By Anthony Piscitelli and Porter Oleson

Election day has arrived, the polling firms have completed their polling, the projection sites have projected, and all that is left is to count the votes. On October 15, 2019, ThreeHundredThirtyEight.com summarized the election projections. Today we are updating those projections with the latest numbers. After the election, we will report on how well the projections did at predicting the actual election results.

We examine eight poll aggregators, one analysis-based projection, and two polling firm models. 338 Canada, Calculated Politics, Canadian Election Watch, CBC Poll Tracker, Lean Toss Up, LISPOP, Too Close to Call, Visualized Politics, @EarlWashburn, @repdonsman456, and @politikstcan each aggregate multiple polls from different polling firms to predict the election. Election Prediction Project and Teddy on Politics makes predictions using an analysis process, which examines data from the public alongside general trends. Teddy on Politics and @kylejhutton made similar ‘gut-based projections’ of the election. Ekos, Forum, and Mainstreet are polling firms who use their data to model the election results.

A glance at the projections shows the Liberals will likely lose significant support from the 2015 election. The Conservatives and the Bloc appear to be the most likely to gain from these losses. The NDP will likely lose some support. The Greens seem likely to hold the two seats they held at dissolution and may gain a few seats. The People’s Party of Canada is projected to have one or zero seats. The projections also predicted most independents will lose their seat. Note, at dissolution, the CCF held one seat and five seats were vacant. These are not listed in the table above.

The table above provides a snapshot of the polling results. The first row shows the highest total any projection places a party and the next row the lowest. The average of all projections is listed next. The final row shows the average of the polling aggregators. It’s worth noting that some of the aggregators provide ranges around their projections, which indicate that a large variety of outcomes are possible in an election like this one where the polling data is so close. Please check the linked website above if you are interested in seeing all the range of potential election outcomes.

The various projections models in Canada, with the exception of the Election Prediction Project,  all use some sort of variation on a regional swing model to predict the election results. Essentially this model uses the previous election results and compares it to the polling data in the current election. So, for example, in a basic swing model if in Saskatchewan a party got 10% of the vote last election and polls now have them at 20%, the projected vote total in every riding for the party would be doubled. If another party went from 30% to 20% the projected vote total for that party would drop by one third in every riding. The website ThreeHundredThirtyEight.com, which was run by Éric Grenier before he went to CBC provides a good explanation of the model he used in 2011.

Some variations in results are generally a result of variations of how the regional swing model is applied. For example, some models provide bonuses for incumbents, whereas others incorporate riding polls. How the polls are aggregated can lead to variation as well. Forum and Mainstreet make projections using their own polls only. The poll aggregators can use different methods to average the polls. Canadian Election Watch, for example, attempts to predict turnout as part of its aggregation of the polling data. Support totals are manipulated to increase Conservatives +1.5% and Liberals +0.5% and decrease the NDP -1% and Green -1%.  The table above shows that while the different methods yield relatively consistent comparisons there are subtle differences. The Liberals, for example, vary between 31% to 33% when examining the aggregators. Note, not all poll aggregators publish their aggregated totals.

If you would like to make your own projection, please feel free to post it in the comments below or post it in the Google Sheet. Share the seat total for Liberal / Conservative / NDP / Bloc / Green / PPC / Independents and what you project as the turnout for the election. In a future blog post, we plan o evaluate the predictions.

*Note projections will be updated as they are made throughout the day. Projections in Italics are not believed to be the final projections of the aggregator.

*Projections updated 6:50am added Visualized Politics, @kylejhutton, and the second 338 Canada model.

*7:20am updated with Canadian Election Watch and Visualized Politics final projections

*8:45am updated with Calculated Politics final projections and repdonsman456 projection update.

*4:30pm updated with Calculated Politics, @repdonsman456 and Ekos final projections.

Federal Election Projections: Oct. 15, 2019

Anthony Piscitelli

Poll aggregating websites and individual polling firms use various methods to project who will win the 2019 federal election. The following table presents a summary of the projections from a number of Canadian prognosticators from October 15th, 2019 (or earlier):

LISPOP, Canadian Election Watch, Too Close to Call, 338 Canada, CBC, @politikstcan, and @repdonsman456 each use aggregating models to predict the election. While Forum and Mainstreet use their own polling data to model the election.

Liberal projections range from a high of 168 to a low of 125. (Note the 168 projection was projected from October 1st). Conservative support ranges from 127 to 150. All of these projections mean a minority government. Four of the nine projections predict the Liberals will have the most seats and five of nine projections predict the Conservatives will end up with the most seats.

The NDP projections range between a low of 24 and a high of 34 seats. The Bloc projections range between 15 and 36 seats. The Greens are projected between three and seven seats. Finally, the PPC is projected to win zero or one seat. Five of nine projections also predict an independent (presumably Jody Wilson Raybould).

It should be noted while these projections are the predictions published, many of the sites also put confidence intervals suggesting wider ranges of possible outcomes.

This is our first post summarizing election projections. If you know of a projection we missed please post it in the comments below or use the contact form to share it.