Making Questions Required -How Online Surveyors Ruin their Results with One Click!

Working at FluidSurveys, I have the privilege of working on a variety of surveys every week. As you can imagine, we deal with a wide range of questionnaires, some being great while others are left needing some work. In this article, we are going to look at a common mistake found in today’s online questionnaires and how you can easily avoid making it.

Problem: Making questions required without giving respondents a way out

As a survey writer, your primary concern should be to ensure that all respondents have the ability to answer to your questions no matter their opinions, behaviours, or attitudes. However, sometimes a particular question won’t apply to a respondent. In this case, the survey needs to include a way to either skip the question or choose an option that reflects a respondent’s lack of opinion on the subject.

For example, let’s say you were asking basketball fans to select their favourite NBA team. Fact is some people who like basketball don’t follow the NBA, and even when only including those who do, some will not have a favourite team. If this survey were done on paper or in-person, it would be easy for these people to just skip to the next question. In an online survey, however, more often than not this question will require an answer before letting the respondent move on to the next screen.

Above we see the problem in live action. Without being able to skip the question or choose to opt-out, respondents who don’t have a favourite team are left in a tough situation. This will lead them to one of two equally destructive options:

1) Drop out of the survey: The respondent will most likely be frustrated they can’t skip the question they have no answer for. This can lead them to drop out rather than give an untruthful response. Not only will this result in a decrease in both your number of responses and completion rate, but it will also taint the image of your organization in the eyes of the disgruntled respondents who feel that you have wasted their time and cheated them out of the survey’s incentive.

2) Spam Your Results: The other option left to the respondent is to provide an answer they don’t actually believe in. Unfortunately, these responses will be a major source of bias in your results. Taking our NBA question as an example, respondents who don’t have a favourite team or don’t even follow NBA are most likely to simply pick the first team they recognize off the choice list. This will only serve to skew your results to the choices near the top.

A rating scale without a way out causes a different form of survey bias. Most often, people without an opinion that are forced to respond to a rating scale will select the choice placed most closely to the middle or neutrality. These responses will spam your results to a more centrist value. The question below, is a perfect example of when this problem occurs:

Sure if you watched Guardians of the Galaxy, this question is perfect. But those that haven’t would most likely have no opinion and be forced to drop out of the survey or select ‘Neither liked or Disliked’. Bottom line, a middle rating is different from having no opinion. One is a scored point on a rating scale and the other is a statement made by the respondent to remove their answer from the data results.

To learn more about balancing rating scales and neutral options, check out our previous FluidSurveys University post, “Odd or Even? –The Ongoing Debate of Neutral Rating Scales”.

Solution: Always have an opt-out choice available

Fortunately, by including an opt-out choice like ‘Not Sure’, ‘No Opinion’, ‘Don’t Know’, ‘Prefer Not to Answer’, and ‘N/A’, you can avoid this issue completely. Though having one of these on your survey may make you feel like you’re giving respondents an easy way to avoid making a choice, the increase in completions and decrease in survey bias will be worth it.

So before clicking that dangerous ‘Make All Question Required’ button, double check that each of your questions have an opt-out choice in them. This will keep your respondents happy and not leave you pulling your hair out when you see your survey’s final response rate and results.

Tip: When using an opt-out choice in a question, you’ll want to eliminate its scoring from your results. Below is a video on how to do just that with the FluidSurveys tool.

What is Your Online Survey Pet Peeve?

We want to know what makes you grind your teeth while you do a survey. Give us the reasons you drop out of a questionnaire before reaching the end and I just might write an article on it! Until next time, happy surveying people!

Don’t have an account yet? Get started by contacting us or visiting our pricing page!

FluidSurveys Presents

Free Survey Q&A

Join our survey & research expert Rick Penwarden as he answers all of your questions every Wednesday at 1PM EST!

Leave a Reply

Your email address will not be published. Required fields are marked *