Mindspot Research Survey Doctor: Questionnaire Design Order Bias

order bias The Survey Doctor says, “the right questions eliminate order bias”

There are many opportunities to introduce response bias into marketing research survey design. The Survey Doctor encourages removing order bias from research questionnaire designs.


The Survey Doctor has recommendations on how to write the right questions. This article is focusing on once you have the right questions, how to design and order them. There are two common order bias mistakes in survey design.


What exactly is order bias? It is a type of response bias in which the order of the choices within a question or the order of the questions biases the answers. This typically happens in 2 ways:


1) Order Bias

This can occur when a respondent is given the opportunity to select answers in multiple-choice question design. Here’s an example from the Survey Doctor:


What color do you consider to be your favorite?


  • Blue
  • Yellow
  • Green
  • Orange

Respondents have a propensity to choose from the beginning and ends of lists.


This is particularly problematic with a list of choices. Often respondents associate first choices with correct choices. In order to remove order bias within a survey question, we recommend randomizing the multiple-choice responses. If you are surveying a sample of the population, let’s say for this example 100 people, ensure that the order changes and all of the colors have the same opportunity of being in any given choice placement by using a randomization scheme.


2) Order Effect

This is often referred to as order bias; however, the order effect is more accurate. When response bias is introduced within the questionnaire design an order effect can occur.  This happens when a question is asked at the beginning of the survey that can influence other responses. An example from the Survey Doctor is asking a person’s income level at the beginning of a survey and then asking a number of price-related questions. This may influence how your survey respondents answer the subsequent questions.


A common error in customer satisfaction questionnaire design is asking detailed questions before obtaining the larger answer. The detailed questions may influence or bias the respondent’s initial response. For this reason, we strongly encourage questionnaire designers to ask overall satisfaction prior to asking questions about various components that could influence the overall satisfaction levels.


If you still have questions we would love to hear them. Call us, we have answers.


by Lynnette Leathers CEO of Mindspot Research and The Survey Doctor


Please follow and like us:
Facebook
Twitter
LinkedIn
YouTube
INSTAGRAM
On Key

Related Posts

Marketing Research Tracking Studies

Marketing research tracking studies are a valuable tool for businesses of all sizes.  This type of marketing research helps you understand your customers, track your progress over time, and make

The Caitlin Clark Effect

A Marketing Masterclass Delivered by a Superstar The world of women’s basketball has been buzzing with the phenomenon known as “The Caitlin Clark Effect.” This rising star, drafted first overall

Generation Alpha

Generation Alpha is the generation of children born between 2010 and 2025. They are the first generation to be born entirely in the 21st century and are growing up in

Thank You

The Team at Mindspot would like to THANK YOU for your support this year and wish you a Wonderful Holiday Season! This year Mindspot has made donations to three special