Avoiding Survey Bias

Continuing our discussion on error and bias, we take a look at the next form of response bias, called survey bias. In this article we will define what survey bias is, identify its major causes in online surveys and discuss how to eliminate this type of error from your study.

Defining Survey Bias

As we already know from our previous discussion on bias, response bias can be defined as the difference between the true values of variables in a study’s net sample group and the values of variables obtained in the results of the same study. A form of response bias, survey bias encompasses any error due to a study’s survey design.
Though survey bias can be found in any form of questionnaire, it is especially prevalent in internet surveys since they are completed privately by respondents. Without the supervision of an interviewer, it is difficult to track participant reactions to the way questions are worded, the selected question types and design, the structure of the survey, or its styling and colouring. Problems in any of these four areas can lead to a dramatic climb in a study’s survey bias and drop-out rates. That is why a researcher must learn how to correctly address each of these four sources of survey bias before they begin fieldwork and start collecting data.

Question Wording

Survey bias can be subtle or blatantly obvious in a question’s wording, and can take many forms. It is the survey designer’s job to remain impartial and avoid writing questions that lead or confuse the respondent. This is usually easier said than done as writers generally create biased questions due to their own lack of knowledge in a subject or ignorance of other people’s perspective on that subject.
To help squash this bias, it is essential to remain neutral in all questions no matter how extreme the topic. It also helps to conduct secondary research to ensure you have a full understanding of the topic in study. Furthermore, a constructive peer review by an expert in the same field of the survey topic will allow you to learn of any problems with your questions that make them confusing or erroneous for the subject of study and its target population.
Beyond this, there are several rules that survey creators should always abide by when writing survey questions. Learn them all and reduce your survey bias by reviewing ‘Get the Most Out of Your Survey: Tips for Writing Effective Questions‘.

Question Type and Design

This type of survey bias includes the selection of different forms of questions (rating scales, ranking, open-ended versus closed-ended) and the options of answers provided to the respondent. The selections made in question types can have significant impact on the responses received. This is also the case for the options researchers provide for participants to choose from.
To avoid running into this type of bias, it is crucial for the researcher to understand the strengths and weaknesses of each question type they will be using. This way the question and its options will not only be chosen correctly but can be tailored in order to provide only the most useful data. For more information on how to use different question types check out FluidSurveys’s video tutorials.

Survey Structure

One of the most overlooked forms of survey bias comes from poorly designed survey structures. Survey structure usually pertains to the order in which the survey questions are revealed to the respondent, but can also refer to the number of questions per page, the survey logic, the survey length, and the introduction and conclusion. Each of these portions of survey structure can contribute to survey bias and drop outs.
As with the other forms of survey bias, the best way to avoid making errors with your survey structure is through studying how modifying each aspect of your survey will affect your respondents’ reactions to each question. For example, putting the most threatening or personal seeming questions at the end of your survey will decrease your number of drop outs. Acknowledging information like this will allow you to construct the best possible surveys.

Styling and Colouring

This section of survey bias includes any form of flare added to a survey design. It can include colour schemes, font styles, logos, videos, sounds and any other type of interactive element. Styling is important to provide stimulus to the participant and avoid respondent fatigue. Moreover, using colours and logos allows respondents to recognize a survey’s legitimacy. However, providing styling can also bias your survey. The fact is, people respond in various ways to different colours and imagery.
It is important to use pretesting to ensure there are no issues with your choice of styling. Ask your pretest team whether they can clearly see and read everything in the survey and if the style used effected how they felt about the survey questions. A rule of thumb for styling is to ensure that the survey cannot be considered directed towards one demographic. Instead any added styling or colouring should make the survey look neutral while still being inviting and professional.

More to Come!

This was our second last article on bias. The next post will discuss respondent bias and the best ways to eliminate this bias before it affects your results. If you haven’t already done so, you may be interested in checking out our previous article on response bias called, researcher bias. Until next time, happy surveying!

FluidSurveys Presents

Free Survey Q&A

Join our survey & research expert Rick Penwarden as he answers all of your questions every Wednesday at 1PM EST!


Leave a Reply

Your email address will not be published. Required fields are marked *