Tips for Overcoming Researcher Bias

In this article we will be looking at the first of three types of response bias. If you are unfamiliar with the terms error and bias or the differences between nonresponse and response bias please check out our former articles on the subject. This article will define researcher bias as well as go into depth on how it occurs and the best ways to avoid it. So, without further to do, let’s get started!

What is Researcher Bias?

Researcher bias is a form of response bias that occurs whenever there is a flaw in a survey’s research design. This systematic error can be caused by problems with various different aspects of a study’s research methodology. Most of these issues arise from lack of planning out a clear research purpose and objectives as well as the absence of secondary research before initiating a study. A lack of planning and overall understanding of the topic being studied can make it difficult to create a survey with the correct list of questions and lead to higher amounts of error.

Combatting Researcher Bias

The following is a list of the common types of researcher bias and some tips on how to avoid each. This list has been altered from the previous schools of thought to pertain to only online research projects:

1. Surrogate Information Error: This form of researcher bias is created by a variation in the information needed to address the marketing problem and the information the researcher is collecting. This problem can occur in various ways, but most often is due to a lack of understanding by the researcher. For example, let’s say the research purpose of a survey was to figure out what type of ice cream is the favourite in a sample. The researcher asks, “Do you like chocolate, vanilla, or strawberry ice cream the best?” Though there are many other types of ice cream, they did not come to the researchers mind. Now instead of measuring the most popular ice cream, the study measures the preference between these three types of ice cream.

The best way to avoid surrogate information error is through exploratory research. Doing preliminary research where you ask several open-ended questions will provide you with the perspectives of several people. This makes it much more difficult to overlook options that respondents may want to select for each question, which in turn increases the value of your results. Beyond this, it is useful to add an “other” option where respondents can provide an answer themselves. Though this makes data analysis a bit more difficult, it will allow participants to provide honest responses.

2. Population Definition Error: This form of researcher bias is produced by the variation between the correct population to address the marketing problem and the definition of the population created by the researcher. Usually, population definition error occurs when the researcher either incorrectly excludes potential participants or incorrectly includes certain participants in the population of the study.

This bias can also be due to a lack of a clearly defined target group for the study. For example, a research problem may require a population that is considered to be of lower economic standings. This population could be defined in various ways; people with low income, people who lack disposable income, or people who have a low net worth after taking into account their property, income and debt. Each of these three descriptions can successfully be used to address the research problem. However, each definition will provide different results for the study.

To avoid this form of bias, it is essential to have a marketing problem that specifically outlines the type of population it wishes to study. It is best for the researcher and the organization with the problem (sometimes the same person) to discuss and agree on the population definition and how it will be targeted. Then the researcher must ensure all reports and presentations clearly specify the description of how the population was defined instead of using ambiguous descriptors like poor, rich, large or small. This will allow for no misunderstandings or errors in the population definition.

3. Sampling Frame Error: Bias that occurs due to the difference between the population defined by the researcher and the actual population being studied through the sampling method. A great example of this is the use of panels (professional groups paid to answer surveys). This sampling method only allows people in the panel to answer the survey and excludes all other potential respondents from participating. Any sampling method that excludes people that fall under the researcher’s target population, in a non-random way, will create sampling frame error. In other words, as long as each member of the defined population has an equal chance of being selected to participate in the survey, the study will have no sampling frame error.

Ensure that this is the case by allowing all respondents the ability to gain knowledge and take part in your survey. For example, a restaurant that is doing a menu satisfaction survey may provide their survey invite as an intercept on their company website. However, using only this method will exclude customers who do not visit the website but still frequently visit the restaurant. In this situation it would be appropriate to add an invite to the survey at the end of each customer’s receipt in order to improve the sampling frame and decrease error.

4. Data Analysis Error: This form of bias is created when raw data is transformed into erroneous research findings. This can be done through inappropriate uses of statistical techniques, leading to the incorrect interpretation of the survey results.

In order to avoid data analysis error, make sure you have a complete understanding of all the statistical techniques you plan to use on your survey’s raw data before you create your question list. With this knowledge, you can write survey questions that complement your planned data analysis. Most researchers that succumb to data analysis error make the mistake of gathering information and then later developing a data analysis strategy. This inevitably involves forcing a statistical technique on the data. Instead, it is easier to create questions that you know will work well with the analysis you have in mind.

FluidSurveys Presents

Free Survey Q&A

Join our survey & research expert Rick Penwarden as he answers all of your questions every Wednesday at 1PM EST!


Leave a Reply

Your email address will not be published. Required fields are marked *