Menu Close

Region of Waterloo Residents Priorities 2019

By Laura Krizan

The results of this poll were based on an interactive voice response survey conducted Friday March 15thand Monday March 17th, 2019. A total of 1003 individuals completed the first question of the survey and 715 completed the entire survey. The survey was designed to aid in the development of the Region of Waterloo’s Strategic Plan for 2019-2023. The Strategic Plan helps Council and staff set priorities and achieve goals, all while keeping the community’s concerns and suggestions in mind. A significant proportion of the Strategic Plan includes hearing input from the public and listening to comments, concerns, etc. so that the Region can set appropriate priorities. The questions in the survey are also aimed to help the Region of Waterloo during the drafting of its Strategic Plan in the future.

The Strategic Plan has 5 focus areas: Thriving Economy, Sustainable Transportation, Environment and Sustainable Growth, Healthy, Safe, and Inclusive Communities, and Responsive and Engaging Government Services. The questions that were administered as a part of this survey were developed by ensuring that these focus areas were kept in mind.

The first question on the survey asked participants about the level of confidence they have in their local government. The results are shown in the chart below, indicating that a majority of citizens (56%) are confident in the regional government to some degree (including somewhat confident, confident, and very confident levels).

The survey was a way to analyze the top priorities that need to be set by the Region of Waterloo for the development of their Strategic Plan. Respondents were asked what they think the biggest priority in Waterloo Region is that the regional government should address.  The results, as shown in the chart below, have been ranked based on the number of respondents choosing a given category as their top priority. The top 3 priorities are: 1) Supporting the development of affordable housing 2) Managing growth, 3) Protecting the environment. 

A significant component of the Strategic Plan focuses on the services that are delivered by the Region of Waterloo, such as public transportation, waste collection, and so forth. In order to better understand the preferences among citizens living in the Region of Waterloo in relation to the delivery of services, the survey asked: “Regional Government must balance the cost of delivering services with taxation. Which of the following would you most prefer for property taxes in Waterloo Region?” 

Results indicated that 19% preferred increasing taxes to improve services while 14% preferred having property taxes decreased. 23% preferred keeping taxes that same and possibly reducing services. The largest proportion (44%) preferred having taxes increased with the rate of inflation and maintaining current services.

This survey was also used to analyze the best ways and platforms to receive public input in the future. Respondents were asked, “If the Region of Waterloo wanted to gather public input or engage you on major issues or decisions, what are the best ways?” It was found that the best ways to gather public input or engage on major issues/decisions in the region are: 1) Online Survey, 2) Social Media, and 3) Telephone Survey. All other options that were included in the survey are listed below.

Ultimately, the survey helped to provide the Region of Waterloo with important information that can be used during the development of the 2019-2023 Strategic Plan. A total of 9 questions were administered, yet the responses that have been analyzed above highlight the most critical results that will be taken into consideration by the Region.

Survey Details

The Interactive Voice Response (IVR) survey was conducted by Laura Krizan; Abby Schlueter; Andrea Volford, and Professor Anthony Piscitelli on March 15th and March 17th, 2019. Throughout the development of the survey, the students worked alongside Lorie Fioze, Manager of Strategic Planning and Strategic Initiatives. The questions that were formulated for the survey focused on supporting the development of the Region of Waterloo’s strategic plan. The survey was funded by the Region of Waterloo to support this initiative. 

Sampling Approach

The sample size was created by randomly selecting Waterloo Region landlines listed in a digital phone book. A sample of likely cellphone numbers was added by randomly selecting phone numbers that were originally assigned to Waterloo Region, according to the Canadian Numbering Administrator. Sampling errors exists as a result of this approach due to the mismatch created by the random dialling of phone numbers as opposed to randomly sampling actual Waterloo Region residents.

Response rate

The survey called 46,912 live lines. The response rate was 1.5%, which is based on 715 respondents who completed the entire survey. All 788 respondents who answered the first three questions were included in the results. It is worth noting that 215 (21%) respondents were not eligible to participate due to being under 18 or not living in Waterloo Region.

Weights

Results of this survey have been weighted by age, gender, and city/township according to the 2016 census. The full weights are posted along with the raw data on OpenIcpsr.org and can be found by visiting: http://doi.org/10.3886/E110225V1  

Margin of Error

Results are considered accurate +/-3.7%, 19 times out of 20. The margin of error on subsamples is higher.

Raw Data

Raw survey data is available on OpenIcpsr.org. The data can be found at: http://doi.org/10.3886/E110225V1  

Disclaimer

The survey results will exhibit sampling error as a result of the mismatch created by the random dialling of phone numbers as opposed to randomly sampling actual Waterloo Region residents. This survey was approved by the Conestoga College Research Ethics Board.

Comparing the Regional Chair Survey to the Election Results

From October 16 to 17, 2018, ThreeHundredThirtyEight.com conducted a survey to assess support for the candidates in the Waterloo Region Chair race. The results showed Karen Redman in the lead but a large number of voters undecided. Ultimately, Karen Redman was successful during the election held from October 22 to 23, 2018. This post looks back to assess how accurate the survey was at predicting the election results.

In the original reporting of the race, we reported a margin of error of +/-4.25%. For simplicity sake, when reporting a margin of error a single value is typically shared. However, the calculation for margin of error actually varies based on the observed proportion. Results close to 50% have higher margins of error than results close to 10%. Forum Research breaks down the rough margins in a handy table by sample size and observed proportions. For example, according to Forum Research’s table, with a sample size of 300, the margin of error can vary between 3.4% (at 10% or 90% proportion) and 5.7% (at a 50% proportion).

In this post, one margin of error per sample is reported calculated at the 95% level (i.e. the results are considered accurate 19 times out of 20). However, in the commentary assessing the accuracy of the results the margin of error for the individual proportions were calculated using an online calculator.

The obvious place to start is to assess the accuracy of the top line results as reported on October 18, 2018. Here we see that the results overall did quite well. The results of each candidate are within the margin of error except for Jan d’Ailly who slightly outperformed. His margin of error was 2.8 percentage points, yet he received 9.7% of the vote, a result 3.0 percentage points above his 6.7% predicted support. The tracking error on this model also performed quite well at 8.4 percentage points. The tracking error was calculated by taking the election results and subtracting them from the survey results, then adding the absolute value of each of these numbers.

The reported results included leaning and decided voters. It is also possible to compare only using decided voters.  Once again in this approach, all results except for those involving Jan d’Ailly are within the margin of error. However, in this model, the tracking error increases to 11.2 percentage points.

A model was also created to predict likely voters. In this model, the results do not work as well. Both Karen Redman and Jan d’Ailly are outside of the margin of error in this model and the tracking error increases to 15.3 percentage points. Interestingly, using only unlikely voters all candidates results are within the margin of error. The small sample size for this group increases the margin of error. The tracking error amongst unlikely voters is 13.1 percentage points.

When the results of leaning and likely voters are broken down by city/township they all fall within the margin of error. However, it should be noted some of these sample sizes are very small creating very large margins of error. It is worth noting with respect to Karen Redman, the Cambridge results were at the edge of the Margin of Error at the 95% level.

The tracking error was lowest in the townships at 2.7 percentage points, followed by Kitchener at 5.9 percentage points, then Waterloo at 9.3 percentage points, and then Cambridge at 18.5 percentage points.

One final comparison was made. The results reported publically were weighted by age, gender, and city/township of residence. However, it is also possible to compare the unweighted survey results to the actual election results. This approach finds all of the results well within the margin of error and a tracking error of 4.8.

Overall the results of the survey were a fairly good predictor of the actual election results. Indeed, even the breakdown by city/township showed results that were a reasonable predictor of the actual election results. However, the likely voter model was a poor predictor of the election results. It is fortunate this model was not used. It is also interesting to note that weighting the variables did not improve the predictive power of the survey.



Three Reasons Why the Regional Chair Poll May Not Predict the Election Results

Election polling results are interesting because we like to use them to predict an election. However, polls represent a snapshot in time so extrapolating results to a future event may lead to faulty predictions. With respect to the Regional Chair race poll, we released this week the need for caution is even greater, as this was a single poll in a low turnout election with a response rate of 0.2%. Three issues warrant particular consideration.

1) The Survey Could be a 1 in 20 Outcome

Public opinion surveys are an attempt to ascertain what the population believes about a topic by asking a small group of people. When reporting a single poll a margin of error is typically given at the 95% confidence level, indicating a range of plus or minus a few percentage points within which the population’s belief actually falls. So, for example, in the case of the poll for Regional Chair, 36.5% of respondents were undecided with a range of plus or minus 4.3%. Meaning statistically the poll predicts somewhere between 32.2% and 40.8% of voters were undecided 19 times out of 20. This caution of 19 times out of 20 is an important one to note as statistically speaking, even if the poll is a perfectly random sample 1 in 20 times the actual result is expected to fall outside of the margin of error (i.e. 1 time in 20 the poll will simply be wrong).

The best defence against this problem is multiple polls on the same topic by multiple polling firms. A single poll showing a result may be an outlier, multiple polls showing the same result is unlikely. Multiple polling firms tackling the same topic also decreases the likelihood that a bias may be built into the sampling procedure (i.e. how people are selected to participate in the poll). Yet, even when multiple poll results show the same result, the polls may not be predictive of future events. The most recent presidential election in the United States provides a case study in the need for caution when extrapolating from polls to a future election.



2) The Undecided Voters

The survey revealed that 36.5% of voters have yet to make up their mind. These individuals were first asked who they would support if the election were held today, then asked if they were leaning towards a particular candidate. Asking vote preference twice is considered best practice and tends to capture even soft supporters for a candidate. The 36.5% in this survey who were undecided, are therefore individuals who are likely quite open to being persuaded by at least two of the candidates. An additional 8.7% said they would prefer not to share their preferred candidate, meaning we have no idea how they will vote. Finally, of those who indicated a preference, 8.0% were only leaning towards a candidate (i.e. when first asked who they would support they said they did not know). These results indicate that over 50% of voters preferences are either unknown or open to changing. Undecided voters could therefore dramatically change the election results. How dramatically? Extrapolating from these numbers, and assuming the poll results are accurate and not an outlier, Karen Redman’s support on Election Day could fall anywhere between 27.6% and 89.3%.

3) The Sample May Not Be Accurate

It is also possible that the results of the poll may be off because the sampling method introduced an unknown error into the process. The sample of landlines was created by purchasing an electronic list of listed landlines from www.telephonelists.biz. Landline numbers were then randomly sampled from this list. In addition, a list of likely cellphones and unlisted landlines was added into the sample. This list was created using data published at www.cnac.ca on the area code and exchange (NPA & NXX). The data from the Canadian Number Administrator can be used to ascertain the first six digits of phone numbers that when originally activated belonged to someone who activated a phone within Waterloo Region. However, with number portability, it is possible to keep your phone number when moving into or out of Waterloo Region. The survey revealed 6% of respondents no longer live in Waterloo Region. These individuals were excluded from the final results. However, there was no correction made to introduce people with cell phones originally activated outside of Waterloo Region. It is impossible to know what percentage of phone numbers were excluded from the survey because they were not included in our sample and it is possible that these individuals support different candidates than those included in the survey.

The second problem with respect to the sample is those who choose to participate in the survey. The response rate was less than one-quarter of a percent. It is possible that the 99.8% of the population that did not complete the survey are different than those who did participate. A total of 86.5% of people called did not answer the phone. It is not possible to know if these people are somehow different. Perhaps the reason they were all busy on a Tuesday and Wednesday evening makes them predisposed to a particular candidate. There is no way to know. Of the 13.5% of people who did answer the phone when called; only 1.5% completed the entire survey. Again, it is not possible to know if these people are somehow different. Perhaps these people exhibited a Shy Tory effect and were unwilling to participate because they did not want to admit their preferences to a (liberal) Conestoga College professor.