With the final seat tally now official, it is possible to evaluate the prediction results. For this analysis, CBC Canada Votes 2021 at 8:00 am on September 25, 2021, is being used. The analysis examines six websites that aggregate polls from the polling firms and then create a model to predict the election are examined (338Canada, Calculated Politics, CBC Poll Tracker, Lean Toss Up, Too Close to Call, and The Signal), three projection websites using their own models (Election Prediction Project, Election Atlas, and Advanced Symbolics) and four polling firms which use their own data to predict the election (Ekos, Forum, Innovative Research, and Mainstreet).
The results show the Liberals with 159 seats, the Conservatives with 119, the NDP with 25, the Bloc with 33 and the Greens with two. Most projections underestimated Liberal support. Mainstreet was predicted exactly 159 seats. The average estimates were missed by five seats. The Conservative support was very close with a mean of 118. Six projections overestimated Conservative support, seven underestimated Conservative support, and the CBC Poll Tracker had exactly 119 seats. The average estimates were very close only missing by one seat. NDP support was overestimated by every projection by an average of seven seats. The Bloc seats were overestimated in ten projections and underestimated in four. The mean of Bloc support was only off by two seats. Seven projections accurately predicted the Greens would win two seats and the average projection was two seats. Five projections predicted the Greens would win three seats and two predicted one seat. Three projections incorrectly predicted the PPC would win one seat.
To evaluate the overall accuracy of the different projections the projection for each party is subtracted from the actual seats won. The absolute value is then taken (i.e. if it is negative it is turned positive) and these values are added together. This gives a value of the error of each projection. (A more accurate method would be to look at each individual riding and see what percentage of the ridings were accurately predicted. This information is not readily available for each of the projections).
The average projection error was 24. The most accurate projection was by Mainstreet who only missed by 12. Mainstreet also had the most accurate projection in 2019. The poll aggregates as a group were the most accurate. Five poll aggregator projections (Calculated Politics, CBC Poll Tracker, Lean Toss Up, Too Close to Call, and @politicstacan) had an error of 14. The most accurate other projection method was Election Atlas with a projection error of 16. Ekos also had an error of 16.
Part of the accuracy of the models used to predict the vote share of each party in the election is due to the accuracy of the projection of the vote share of each party. The next comparison examines how accurate the vote share was projected. The average error is calculated in the same way as the previous analysis, with the absolute value of the difference taken for each party.
The average error was 10.1 percentage points. The most accurate projection was from The Signal with an error of 6.2 percentage points. Too Close to Call was just behind at 6.5 percentage points off. All but one of the aggregators had better projections than each polling firm. Calculated politics had the same tracking error as Mainstreet at 10.7 percentage points. Mainstreet was the most accurate polling firm once again. On average, the projections underestimated the Liberal, Conservative, and Bloc party support were underestimated. The NDP, Green, and PPC support was overestimated by the projections.
Every projection accurately predicted a Liberal minority government with the Conservative party in second place. Seat projections for the Bloc, Greens, and PPC were also reasonably accurate. The NDP seat projections were the biggest miss with every projection overestimating the projected seat total with an average error of seven seats. The Liberal party support on average was overestimated by five seats on average. A post-election analysis of why the NDP was down in the projected vote and seat count is warranted to improve future projections. Overall, the projections were very accurate, and the projection modellers should be satisfied with their predictions.