Odd or Even? –The Ongoing Debate of Neutral Rating Scales
No, we’re not talking about roulette at the casinos! We’re talking about the number choices provided on a survey question. Since the creation of the rating scale, there has always been disagreement on whether closed-ended survey questions should have a neutral category or should force respondents to take a side. By creating a scale with an odd number of categories, a researcher will be leaving a mid-point, which acts as a neutral option for respondents to select. Without it, respondents will be forced to pick an option on either the lower or higher end of the rating scale. There are different schools of thought on whether an odd or even number of categories is best, but in this article we will discuss the strengths and weaknesses of both.
The Power of Fence Sitters: Why Neutral Responses are Important!
In most situations, having an odd number of categories is usually the more effective way of gathering accurate data. The fact of the matter is many people are legitimately neutral on a subject. Forcing respondents to answer a question on an even scale will bias your end results as truly neutral people must select a category that does not truly represent their opinion.
The question below is a typical likert question asking people to provide their level of agreement with a statement on potato chips. As you can see, the scale has an odd number of categories with a neutral point in the middle. Each category has been allotted a score number, the higher numbers favouring barbeque and the lower numbers favouring salt and vinegar. Taking away the middle category would force respondents to choose between the two flavours, even if they truly do not have a preference of chip type.
Many argue that they can avoid the survey bias caused by an even number of categories by providing an opt-out category for neutral respondents. This means adding a ‘No Opinion’ or ‘N/A’ option at the end of your scale that, if selected, will eliminate that response from the collected answers. However, this would actually result in a more extreme bias in your data analysis. Let’s take a look at why.
Say we reworked our potato chip question to be as follows:
The problem lies in the difference in scoring. The power of neutral responses has been eliminated from the data with the label ‘No Opinion.’ But with this change, we will see a dramatic effect on the intensity of the sample’s overall preference. Let’s say the results for our survey question were as follows:
Strongly Agree = 13 responses
Agree = 22 responses
Neutral = 32 responses
Disagree = 10 responses
Strongly Disagree = 5 responses
If we calculated the average score of our respondents’ agreement with the potato chip statement we would get 3.3 (3=Neutral), but if we used the 4 category rating list and moved all neutrals into the ‘No Opinion’ category, our score would transform to be 2.9 (3=Agree). Evidently, this difference in scoring completely changed the outlook of our results. Omitting neutral answers biases the data to be more in favour of barbeque chips, when in actuality, our sample is much closer to being neutral on the subject.
The other problem with using an opt-out strategy is that some neutral respondents feel obligated to pick a side if there is no midpoint on the scale, whereas others will choose to opt-out. Without being able to control this, the researcher would be at the mercy of respondent discretion, having answers for the same opinion (neutral) in different categories of the rating scales. Instead, the safest route is to keep any normal rating scale odd and therefore safe for the fence sitters.
When to Kick ’em Off the Fence!
However, there are times when an even rating scale can be more effective than one with a neutral point. Sometimes a decision maker will be faced with the dilemma of choosing exclusively one option over another. Say our potato chip company has to discontinue one of its two flavours. As a researcher, we have to acknowledge that the results must fall on one side or the other of the scale. In this case our neutral respondents truly wouldn’t matter to our results. Here is an example of how we can set up our question and its scoring:
In the question above we see that there is six categories and an opt-out choice. The reason for the extra categories is that we want people to make a selection even if they only slightly prefer one brand over another. Making categories with adjectives like “Somewhat,” “Moderately,” or “Slightly” allows fence sitters to take a stand on the question without completely forsaking the other side. Of course, we still have to add an opt-out choice, because we do not want to force a decision on our respondents who are truly neutral.
Moreover, by giving six options of varying degrees of preference, our results will give us both the ability to see the absolute number of respondents who prefer each flavour, as well as a closer measure of the intensity of their preference. The absolute number of respondents’ preference would be calculated by adding the total of respondents that fall on either side of the scale, whereas intensity of preference would be measure by adding the sum total of all response scores. In our example above, the more negative the number means a larger preference for salt and vinegar and the more positive the number means a larger preference for barbeque.
Where do Your Respondents Stand?
We learned that odd and even rating scales have different uses. Odd scales have neutral categories, which serve the dual purpose of eliminating the bias caused by fence sitters and providing more accuracy to your data analysis. This makes odd numbered rating scales the standard for surveying. However, when a decision maker finds themselves in a fork in the road, having to choose between two options in order to move forward, it can make sense to have an even number of categories in your scale.
Latest posts by FluidSurveys Team (see all)
- It’s All About Timing –When to Send your Survey Email Invites? - April 1, 2015
- Making Questions Required -How Online Surveyors Ruin their Results with One Click! - February 26, 2015
- Quota Sampling Effectively -How to get a Representative Sample for Your Online Surveys - February 19, 2015
- The Power of Repetition -How to Measure Your Organizations Progress with Survey Research - February 6, 2015