Maximize Response Rates and Minimize Bias with the Proper Survey Structure

As we all know, a lot of work goes into creating a survey questionnaire. Usually researchers put most of their emphasis on selecting the correct question types and crafting their question wording carefully. Though this is very important, it should not lead to the neglect of implementing a proper survey structure. Survey structure refers to both the grouping of your questions and order they appear on your questionnaire. In fact, a large portion of response bias and respondent drop outs come as a direct result of poor structuring. In this article, we will discuss the best ways to order and categorize your survey’s questions to entice more and better quality responses, getting you the data you need.

Categorizing Your Questions

The first step in correctly structuring your survey is to group your questions into different categories. Having a questionnaire divided with categories will result in an intuitive survey that is easy for participants to navigate through. What’s more, the researcher can improve the survey taking experience further with this type of survey structure by providing an additional explanation at the beginning of each category. This gives the respondent an idea of what to expect in the following section and the knowledge that the survey is switching topics.

For example, if I were asking for feedback on my company’s new product I could divide a questionnaire into the following sections: design, function and support. Then, I could begin the design section with: “The next set of questions was created to measure the overall design of our product. Based on your own preferences, please answer the following to the best of your ability.” This short section heading not only provides the respondent with the knowledge that the survey is changing categories, but also identifies the purpose of the questions to follow. Being transparent with the motive of each category will build trust with the respondent and allow them to enter the right mindset for your questions. This will result in higher response rates and better quality answers.

How to Divide Your Survey into Categories

Before even beginning to formulate and word your question list, you should divide your questionnaire into categories. A properly designed survey should be created based on a well formulated research purpose and list of research objectives. The research objectives represent the information you are looking to gain from your research project. When developing a survey’s question list, it is important to ensure each question directly ties into one or more research objective. Done correctly, it should be easy to divide a questionnaire into categories based on its different objectives.

For example, if I were to create a website feedback survey, my research purpose and consequential research objectives may look something like this:

Research Purpose: Measure the level of visitor satisfaction for my website and collect feedback in order to better meet my visitors’ needs.
Research Objectives: Measure the level of visitor satisfaction and collect feedback for each of the following aspects of my website:
1) Usability
2) Quality of Information Provided
3) Visual Design
4) How it Compares with Competitors
5) Overall Quality

With this list, I can know create several focused questions for each research objective. If you are interested in learning more about creating research objectives, check out our article “Creating the Correct List of Questions for a Survey.”

Question Ordering

Dividing your questionnaire into separate sections is only the first of two steps in creating a proper survey structure. It is also crucial to design your question list to be in the correct order. A well-ordered question list will dramatically decrease dropout rates and coax respondents to be more truthful with their answers. The following list depicts the order different questions should be placed in a survey:

1) Screening Questions: These questions decide whether a participant is valid for the survey, and eliminates any unwanted respondents. Screening questions are placed first to ensure that your sample group is correct and unwanted responses will not bias your results.

For example, if the researcher is analyzing the study habits of post-secondary undergrads they might ask “Are you currently enrolled at a college or university? -YES (Continue) -NO (Terminate Respondent)” This will eliminate any non-student respondents from your data.

2) Easy to Answer Questions: The beginning of every survey should include questions that do not require a lot of effort on the respondent’s part. Respondents are most likely to drop out of the survey early on. As they continue to answer questions, the time they have invested and progress they have made strengthens the respondents’ resolve to finish the survey. So make sure to save any questions that require complete sentences, use of memory, critical thinking, or providing personal information for later on in the questionnaire.

Most easy questions are closed-ended and are not complex. For example, “Out of the following types of ice cream, which is your favourite? (Vanilla) (Chocolate)…” Ice cream is familiar to your respondent and you provide a clear list of options for them to choose from.

3) Difficult to Answer Questions: Now that the respondents have warmed up on the easier questions, it is time for a few difficult questions. To further entice respondents to answer these questions, include a progress bar in your online survey. By seeing the progression they have made and the amount of survey left to complete, respondents will be motivated to fill out difficult questions in order to finish.

Typically, the more effort that the respondent has to commit in order to answer a question the more difficult it is. The perfect example would be an open-ended question asking for specific information. For example, “Please list your top three most expensive purchases in the last five years and describe why the purchase was important to you.” This question requires the respondents to use their memory and elaborate on their answers in their own words.

4) Socially Loaded Questions: The final questions of a survey should be ones that are more personal in nature or can make the respondent uncomfortable. These questions are most likely to cause dropouts because respondents feel that there is a socially desirable answer that they should provide beyond the truth. Though these questions are the reason for most incomplete surveys, placing them at the end of the questionnaire will give the respondent motivation to finish.

Some of the most prominent socially loaded questions in surveys come from demographics. This includes questions regarding annual incomes or education, which have a strong social weight in our culture. But socially loaded questions can also be behavioural in nature, like asking about criminal activity or exercise and dieting habits. For more information on this form of question, check out our article “Tips for Avoiding Respondent Bias.”

Show Off Your Improved Survey Structure!

Now that we learned how to build an expert questionnaire, it is time to put our knowledge into practice! Start implementing proper categories and question order, and you’ll be gaining higher quality, actionable feedback in no time!

FluidSurveys Presents

Free Survey Q&A

Join our survey & research expert Rick Penwarden as he answers all of your questions every Wednesday at 1PM EST!

Leave a Reply

Your email address will not be published. Required fields are marked *