Menu Close

Tracking Public Opinion: Ontario Election Wcalc Results

By Suhani Singh

The previous Wcalc blog discussed the results generated from compiled federal polling data from November 7, 2015 to May 4, 2019. We discovered that our results were not consistent with other polling trackers in terms of calculating popular support percentage for the People’s Party of Canada and “Other Party” support which were overestimated. However, it did a satisfactory job in estimating Liberal, Conservative, NDP, Bloc, and Green Party Support. Since the federal polling data is just an estimate at this time with no election results to compare to, we wanted to test Wcalc once more with 2018 Ontario election polling data before we completely disregarded the Wcalc program for our polling calculation purposes.

We ran the overall Ontario polling data collected from March 11 to June 6 through Wcalc. The goal was to compare the Wcalc results to the real election results to test the programs ability to model changes in public opinion toward provincial parties over a short period of time. The expectation was that Wcalc would create a clearer picture of election results, as it proved successful at predicting the 2015 federal election results.

For the purposes of running the data in Wcalc, we removed the Green Party and “other party” polling data from Ipsos and Leger because the Green Party was not asked as a separate answer for respondents, thus inflating the “other party” category and giving us no Green Party data.

 The table below compares the actual percentages of the popular vote from the 53rd Ontario general election with Wcalc the final predicted results based on compiled polling data. It is clear that the Wcalc results were significantly different from the election results.

Party Election results Wcalc results
Conservatives 40.6% 36%
Liberal 19.3% 14.9%
NDP 33.7% 41.7%
Green Party 4.6% 5.7%

The Conservatives and NDP had the most shocking results as their numbers were far from reality and seem like they should be switched with one another. The Conservatives were off by 4.6% scoring less than what they actually received in the election while NDP support was inflated by 8% in Wcalc from the real numbers. Had these results from Wcalc been released before the election to predict the outcome, it would have wrongly predicted an NDP victory.

Comparing our predictions to CBC Poll Tracker and Wikipedia, we can see how different and skewed the Wcalc results are when predicting election results based off of poll averages. The two other predictions were much closer to the actual results of the popular vote as indicated in the chart below.

The time series generated by Wcalc is shown in the graph below. It is evident from the graph that the NDP takes a bit of a jump during the campaign period, followed by a slight drop, and then emerges on the top with 41.7%. Conservative support declined a little and then gradually increase to 36%. The prediction for Liberal Party was 4.4% lower than the actual results and 1.09% higher for the Green Party. The Green Party difference was not significant and fell within a +/-1% margin of error.

The scatter plot graph for Ontario shows a neck to neck race between the Conservatives and the NDP. Conservatives can be seen polling close to 40% with NDP just slightly under. Wikipedia’s graphical analysis predicted results closer to actual election results with a margin of only +/-2%. Wikipedia used local regressions for smoothing the data, therefore, projecting results closer to the actual election results.

The data from WCalc did not match the election result for PC and NDP. The Dyad Ratio Algorithm that Wcalc is based on overestimated the trend of NDP support and while underestimating PC support.

These results were disappointing and have caused ThreeHundrededandThirtyEight.com to reconsider the use of Wcalc for predicting elections. While the research evidence suggests Wcalc is effective at modeling policy mood, we will no longer be using it to predict election results.

Tracking Public Opinion: Federal Wcalc Results

By Natalie Pikulski

The original plan for ThreeHundredThirtyEight.com was to use Wcalc, a program developed by James Stimson to model changes in public support for provincial and federal parties in Canada. Wcalc uses the Dyad Ratio algorithm which is explained in greater detail in a previous ThreeHundredThirtyEight.com post. The Wcalc program worked very well for a trial run on the 2015 Federal Election but unfortunately more recent attempts to use the program have shown flaws in this approach. It appears the local regression approach (as used by Wikipedia), is a more effective method to track opinion changes related to elections, whereas Wcalc may be better suited to tracking policy mood. This blog post examines what went wrong when using Wcalc to track Canadian public opinion since the 2015 federal election and the following post examine the problems associated with using Wcalc to predict the 2018 Ontario provincial election.

We began this analysis by collecting all usable individual Canadian federal polling data from November 7, 2015 to May 4, 2019 that were reported by CBC Poll Tracker and Wikipedia. Usable data was based on working links to reports that are available online as of May 2019 and polling questions that are consistent or similar to questions asked by other polling organizations. Voter intention percentages and sample sizes were based on all leaning and decided responses in each poll. 

In an Excel spreadsheet, we tracked support for six party categories (Liberal, Conservative, NDP, Bloc Quebecois, Green, and Other) with the PPC added in 2018. However, there were notable issues that created some problems in our tracking: 

  • Mainstreet did not have an “other party” category until 2018 which resulted in 2 years’ worth of polls that were not able to be entered in the “other category” when using the Wcalc program as blank values are not accepted. 2018 onward, the organization included an “other party” category and eventually included a “People’s Party of Canada” option since the party emerged, allowing us to use all the data from this time forward.
  • Abacus Data conducted rolling polls in 2019 however the reports from March 4 to March 10 did not include detailed breakdown tables that we could pull “other numbers from”. The reports only provided graphs that indicated overall numbers for each main party’s support but did not feature a graph line for the “other category”. We needed to remove these polls before running the “other category” through Wcalc. 
  • Ipsos did not consider the Green party a separate category for any of its polls despite the Party polling at similar levels to the BQ, which was included by Ipsos. All the Green Party support was likely placed in the other category which inflated the category significantly when compared to other polling organizations that had Green as a separate category. These high “other” polls and blank Green polls needed to be removed before running the data through the Wcalc program. Ipsos only had one poll which included the Green Party and the PPC as a separate category, but this poll was not usable for our purposes as the Wcalc program requires at least 2 similar data sets to compare results. 
  • All Nanos Research polls were entirely removed for our purposes since the question asked had respondents rank their top two choices for voting intention rather than one choice like all other organizations. The difference in questions was not consistent with other polls and therefore not usable for the Wcalc program.

After running the individual polling data through the Wcalc program, the results showed some close similarities to other polling trackers for the Liberal, Conservative, and Green parties and a slightly larger gap between the NDP and Bloc but still within decent range. The main issue encountered with the program was the significantly higher score of the PPC and “Other Party” categories that scored much higher in Wcalc when compared to other polling averages. The average below are as of early May 2019.

The Wikipedia poll tracker uses a Loess regression method to calculate the public opinion poll averages over time. While most parties had similar results when compared to our Wcalc calculations, the results for the PPC were much higher in Wcalc while the NDP was lower. It should be noted that both Wikipedia and CBC poll trackers included the Nanos polls that we had removed due to their inconsistency with other polls. This may be a contributor to the differences, along with the limitations of Wcalc. 

Overall, Wcalc remains better suited for long-term tracking of public opinion polls that are not as susceptible to sudden changes like political/election polling are. The smoothing effect in this program assumes sudden changes are likely due to sampling errors which is less effective for polls that may change drastically in a short amount of time. The gaps in our results compared to others indicate this program may not be as reliable as others for our purposes. These gaps are even more pronounced when examining data during the 2018 Ontario election, which will be analyzed in our next blog post.

The Narrative of the 2018 Ontario Election: Regional Analysis

By Suhani Singh

The previous three blogs in this four-part series have explored the voting intentions of Ontarians during the 2018 general election. The voting trends observed overall, in each gender, and each age group were discussed in the previous blogs. This blog examines regional voting preferences noting trends and differences. Polling data was categorized based of the common regional breakdowns including:

  • Toronto 416
  • Greater Toronto Area 905
  • Eastern Ontario
  • South Central Ontario
  • Southwestern Ontario
  • Northern Ontario

In order to determine the number of seats won by each party in a given region, the winners from each riding were added up based on CBC tracker results.

The Toronto 416 region represents the city core and close surrounding areas. The Conservatives polled between 35% – 45% during the pre-campaign period closely followed by the Liberals (30% – 40%) while the NDP remains low at around 20%. NDP support rose to 25-45% support during the campaign leading the Conservatives and NDP to be in a close race. Liberal party support significantly decreases over time falling between 15% – 30% before the election. This was a dramatic drop since the Liberals were polling similar to the Conservatives in the pre-campaign period.

In the 416 region, the PC and NDP tied winning in 11 seats each and the Liberal party claimed 3 seats.

In the Greater Toronto Area 905 region, The Conservatives polled rather high (between 40% – 50%) during the pre-campaign period followed by the Liberals (at 20% – 30%) and the NDP (15%-25%). The NDP rose significantly during the campaign period to 25%-40% resulting in a close Conservatives and NDP race. Liberal support decreases over time dropping to 15% – 25% close to the election.

Though the polls show that the NDP gained significant support in the month of May, the Conservatives won a landslide in the 905 area taking 31 seats. The NDP won only 4 seats and the Liberals failed to win any seats despite consistent levels of support. These results reveal how the Ontario first-past-the post-election system can obscure significant support for parties in some areas.

In Eastern Ontario, Liberal support stayed stable between 20% – 30% support throughout the pre-campaign and campaign period. NDP support rose slightly from 15%-25% during the precampaign period to 20% – 35% closer to the election. The Conservatives remained on top throughout despite a falling from 40%-55% to 30-50% as the election approached.

The Conservatives emerged as the winners of 14 seats, while the NDP won 2 and Liberals won 3.

In South Central Ontario, Conservative support had a slight fall over time but largely stayed on top of the other parties polling between 30 – 50% near the election. The NDP saw a significant growth in their supporters, starting from the 20% range and rising up to 30% – 55% toward the end.

Polls indicated a tight race between The Conservatives and NDP as the election approached with the NDP eventually claiming victory in the region winning 7 seats while the Conservatives gained 4. Liberals saw a drop in their support (5% – 20%) over time polling similar to the Green Party and falling below the national average (15%-25). Neither party won any seats.

In South-western Ontario, Conservatives polling between 40% to 55% during the precampaign period but saw a drop to 30% – 45% closer to elections. The NDP went from 30% at the start to 40% – 55% closer to elections making it a close race between the two parties. Liberal support was low (15%-20%) to begin with and saw further drop to 5% – 15% over time. Green Party support was somewhat low (<10%) and stable in this region but the party saw won its first seat in Guelph. Liberals failed to win any. PCs won 12 and the NDP finished with 8.

Conservatives and the NDP start polling similarly in Northern Ontario between 20% – 50% but the PCs saw a decline to 15% – 40% in late May while the NDP polled 30% – 60%. It should be noted the support range for individual parties was very large, likely due to this region having the smallest sample sizes. However, polls were relatively reflective of the results which saw the NDP win 8 seats and the PCs 4 seats. The Liberal Party saw some decline from 20% – 25% in precampaign period to 5% – 25% near the election They managed to win 1 seat in this region.

Overall, the PCs won 76 seats, NDP won 40, and the Green Party one 1. Liberals managed to win only 7 seats, one seat shy of maintaining their official party status. This Ontario 2018 elections blog series, looked at the different trends in voting intentions among Ontarians. Men and women presented difference in their choice of preferred party. Party choice also varied from region to region and from generation to generation.