People seem fond of agree/disagree scales. How many times have you taken a survey that was full of agree/disagree responses?
I suspect this fondness is in part due to the fact that it doesn’t take much effort to come up with the response options. Or maybe it’s because survey creators assume the words have universal meaning.
In any case, I see agree/disagree used most often in grid or matrix style questions. One of the most common ways to fit several questions into a grid is to convert the individual inquiries into statements, and apply agree/disagree response options. For example, a question such as “How helpful was this blog posting?” would need to be converted into a statement to agree with “This blog post was helpful.”
And by converting the question into a matrix with agree/disagree, the surveyor might then list out all their previous blog post topics and ask you to agree/disagree on the helpfulness of each post. This approach is dangerous because it induces acquiescence response bias—the trend that people offer more agree answers than disagree answers solely by virtue of the question format. That is, the response options push people toward “yes” answers because we all have a tendency to be polite, respectful, and agreeable—especially toward people in positions of authority and power, like researchers, employers, and teachers (all people who tend to create a lot of surveys).
Instead it’s best to give questions their own space and focus on the construct of interest, such as understanding how helpful a blog post is, how satisfied an employee is with the benefits plan, how challenging a student finds a particular course, how clean a customer finds a restaurant to be, and so on.
What do you think? Do you agree to disagree? What challenges have you had in pinpointing the key construct in your questions? Share your stories with us, and we’ll share our tips with you.
