By Ann Ray
On July 15, 2006
Updated October 9, 2012

When you ask a question, do you offer the answer they need?

I love my new little laptop, so when the manufacturer popped up a survey invitation today (time delayed 3 months from purchase) I was happy to provide feedback. The survey was primarily about how I'd been using the system, which should have been easy questions to answer. However, as I went through the survey many of the questions weren't easy for me, because 9 of the 19 questions were missing my preferred answer.

Two representative questions are:

Which best describes your game-playing habits?
   Playing online PC games
   I do not play games

Which of the following music-related activities have you performed on this PC? (Select all that apply)
   Transferred music to a portable player device
   Purchased and downloaded music from an online Service
   Streaming music through the Internet (Internet Radio)
   Created a music CD
   None of the above

The first scale's issue is fairly straightforward—it's missing options for off-line PC games and console games.

The second scale's problem is more subtle. It doesn't take into account multi-computer users or households. While I had not purchased music on that particular PC, I had copied my music library from my desktop, including protected audio files purchased online.

When respondents encounter questions like these, they have a few options. If it's a required field, they either provide the closest response or abandon the survey. If it's not a required field, they can leave it blank or try to find the closest answer.

Many times when I see this issue in a survey, the closest response—including NA or a blank—is not close at all. When a respondent is asked a question but not permitted to answer, they feel a little bit of frustration. When this problem or others recur in the survey, that frustration builds, increasing the odds they'll abandon the questionnaire or transfer some of that frustration to your brands.

Note that a "Comments?" field at the end of a survey acts as a partial safety valve. The respondent gets to elaborate on the issues they care most about, reducing their frustration, and you get valuable feedback for tuning the questionnaire. However, if the survey was a one-time exercise, you've still lost your opportunity to provide all the respondents with a full set of answer options—an important part of what makes surveys quantitative research rather than qualitative.

Filed under:
Questionnaires