Click anywhere to close

What makes a Survey Question Confusing?

Posted: May 28, 2015

What makes a survey question confusing?

Confusing NPS SurveysA survey question is confusing when the intent or meaning of the question is unclear.  This leads the user to a feeling on uncertainty about how to answer.  By contrast, when a question is clear and to the point, the user is able to quickly decide upon the right answer and provide it back to you. The key is that the user already knows the answer to the question.  You as the survey administrator are simply trying to get that piece of information from them.  Make it as easy as possible for them to retrieve that piece of information.

What is an example of a confusing survey question?

Here is an example of a survey question that elicits confusion and misunderstanding. Is it possible or impossible for the Seattle Mariners to ever win the World Series, or do they lack the required talent to do so. There are several things wrong with this question.  Let's dive into them. First is that this questions is actually asking two questions.  The initial question is whether or not the Mariners can win the world series.  The second question is whether or not they have the talent to win the world series.  The user cannot answer both of these questions.  The answers may be conflicts.  The user is then confused and does not know how to respond.  Be careful that you don't ask two different questions at the same time. Second is that you are asking a question with an endless time probability.  We don't have time here to do a few analysis of endless time probability questions, but suffice it to say that endless time probability questions are opinion questions where both answers are valid when given enough time.  In this case we are asking the user if the Mariners will EVER win the World Series.  This is a difficult question to answer and the user may respond that anything is possible when given enough time.  This answer does not help your research.  If however you were to limit the time range to say, the next 5 years, the user can provide a more appropriate answer to the question. Third is that the question presents a Survey Trap.  By asking whether or not the Mariners lack the required talent to win the World Series we are trapping the user into answering in regards to their talent level.  However your are really asking about whether it is impossible or possible for them to win.  Since the user is focusing on the talent level, the user will now be confused when the answers are provided and relate to whether or not winning the World Series is possible or impossible.

How do you improve confusing survey questions?

Using our example from above, let's look at how we could have re-written it. Does it seem possible for the Mariners to win the World Series within the next 5 years? This question is much better.  it is easy to understand and the user can provide an opinion based response in a decent amount of time.  Let's look at how we improved this confusing survey question. First is that we simply took out the second part of the question about their talent.  In this case their talent level was implied.  Meaning, their talent level will have a direct correlation to whether or not they can win the World Series.  there is no point in asking both questions. Second is that we added a timeframe of 5 years to the question.  As the timeframe gets longer there will be a diminishing value in the data that is contained within that timeframe.  Keep the timeframe appropriate and concise to better fulfill it's purpose. Third is that we removed the survey trap by focusing the question on whether or not the Mariners will win the World Series.  The user knows exactly what is being asked and how to answer.

Conclusion - Identifying and Fixing Confusing Survey Questions?

We learned about what can make a survey question confusing.  These are thing such as asking 2 questions, presenting a survey trap and even opening up the question to endless time probability issues.  These are all common mistakes and easy to make.  However with a little thought and simple editing they can easily be removed from your survey.

Copyright © 2017 Dimbal Software - All Rights Reserved
Dashboard | Software Modules | Support | Blog | Article Index

Links | NPS | Polls | SEO Scans | Test Plan | Triage