In the final post reflecting on the 2018 municipal elections, how the different surveys performed at predicting the elections will be explored. Between August and November 2018, 20 races were polled examining municipal races in Winnipeg, Toronto, Brampton, Hamilton, Waterloo Region
To evaluate the accuracy of each firm the poll conducted closest to the election for each race was compared it to the final results. Two questions were then asked:
- Did the poll accurately predict the winner of the election?
- How many candidates vote totals fell within the poll margin of error?
Some polls allowed for an ‘Other Candidate’ option, rather than listing all the candidates running. When this occurred the candidates not listed were collapsed into a single category for comparison purposes. When examining question 1, the surveys did quite well with 15 of 17 elections using IVR successfully predicting the election winner. On question 2, results were not great. Only 50 of 103 individual candidates predicted vote totals fell within the margin of error at the 95% level.
As mentioned the polls did well at predicting the winners. However, when examining the top two vote finisher in each race of the 17 races, 7 of the results were outside of the margin of error for both candidates. Conestoga’s poll for Kitchener Mayor, Forum’s poll for Toronto Mayor and the Mainstreet poll for Brampton Mayor all had the top two candidates fall within the margin of error at the 95% level. Most of the results outside of the margin of error were very close missing by only a few percentage points. Missing this closely could easily be the result of undecided voters breaking differently than those who had made up their mind or a result of people changing their mind.
When examining the results in greater detail only two polls incorrectly predicted the winner. Summarizing the results of each of the polls closest to the election are analyzed by the individual survey firm’s accuracy and average tracking error in the table below.
Research Co conducted five polls related to Vancouver and Surrey British Columbia municipal election races. However, the Surrey poll did not explore the race for mayor, instead the poll gaged support for each candidate individually. It was therefore not possible to include the Surrey poll in our results. With respect to the five polls conducted for the municipal Vancouver race, when examining Research Co.’s final poll they identified the winner but only one of the top two candidates results fell within the margin of error.
Probe Research Inc. conducted two CATI to web polls for the Winnipeg Mayoral race that both identified the winner. Examining the final Probe poll they predicted the correct winner but were outside the margin of error for three of the eight candidates polled. Two additional candidates cited in the poll did not appear on the ballot on Election Day. It is worth noting that Probe’s poll was in the field a month before election day, which stands in contrasts to the Forum, Mainstreet, and Conestoga College polls which were conducted within a week of election day.
Overall the results of the surveys were a decent predictor of the actual election winners. We reviewed 17 IVR polls, a Cati to Web poll and an online poll. In total 17 out of the 19 polls successfully predicted the winner. This is a good result, however, when examining the individual margins of error the results were not as strong. It is important to recognize that there are lots of reasons the results can be off by a little bit. These polls are snap shots in time taken before the election, meaning people can change their minds. The polls try and capture voters’ perceptions but it is possible the people who respond to the poll are not reflective of the people who will actually vote on Election Day. Overall, these polls provided a relatively accurate perception of the state of the races polled.
*This article has been update to correct the type of poll conducted by Probe and to note the date Probe’s polls were in the field