Online, phone and in-person surveys: A, B, C or all the above?

In the past three years, I have observed an uptick in the number of counties, cities and towns soliciting feedback on their solid waste management and recycling programs from residents and businesses through public opinion surveys. This is heartening because one of the best ways to determine whether a solid waste management program works well is to ask community members.

If you are developing a survey for your constituents or customers, do not rush the process. Surveys take time to develop and iteratively test and improve. If you have ever taken a poorly designed public opinion survey, it usually is abundantly clear when a question is leading. In these instances, you could be forced to choose among three suggestive options without an option to skip the question or express no opinion.

Surveys are strategic tools to gather feedback from a population. They can be particularly useful for gathering opinions from those who might not typically comment on public services or attend meetings. They can be designed for government or business needs, such as helping to brainstorm solutions to common problems. Surveys also can help weigh policy updates, determine fees for additional services or prioritize new infrastructure to advance waste diversion goals.

Frequently, jurisdictions offer a mix of in-person, online and phone surveys. If you are deciding which method (or methods) to employ for a survey, how you will use the information gathered—among other considerations—should influence your choice.

In-person, online and phone surveys:

• can be designed to be anonymous to increase candor;
• can be publicly reported in the aggregate if anonymity is a challenge because respondents’ identities could be important if answers are tied to specific services or if participation must be tracked;
• can be designed and executed by a third-party research firm to deliver independent, objective results; and
• should require consent, which is essential, and enable participants to opt out of completing the survey.

Online surveys:

• should be advertised through available means, including print, online, TV, radio and social media outlets; shared by invitations; mentioned in newsletters; heard during briefings; and seen on printed posters and banners where people congregate, pass by, or where waste management is topical, such as at recycling and compost drop-off areas;
• could produce lower completion rates than other methods but reach a large audience because outreach is infinite;
• could produce results affected by selection bias given their voluntary nature and how they are advertised;
• should include screener questions to dissuade unintended respondents and “ballot box stuffing;” and
• can be considered a first step in cases where non-random data are needed, with phone surveys used to gather randomized data for comparison.

Phone surveys:

• can be scientific and fully randomized to generate statistically valid results with high confidence intervals and low margins of error, minimizing the impact of selection bias;
• can call landlines and mobile phones, ensuring thorough geographic coverage;
• enable quick gathering of responses;
• can be administered by a remote survey facilitator, typically at a call center where employees can be trained on specific subjects, including accents;
• do not require advertising—it might even be discouraged to avoid “priming” respondents; and
• can include survey logic so that callers ask the most important questions.

In-person surveys:

• can be delivered by a survey facilitator at a designated location or at various locations to gather a diverse sample;
• will result in fewer responses because of the number of facilitators available and how long they can work;
• are ideal for collecting answers with more context;
• enable trained survey facilitators to collect nuanced information by observing body language and facial expressions, which is not possible with other survey methods; and
• can be used for longer surveys because respondents could be more willing to complete them, depending on the topic.