By Ann Ray
On March 25, 2008
Updated August 9, 2013

Watching for overly broad questions

Recently I completed a telephone survey, and in the course of the 22 minute conversation (estimated at 12-15) I was asked:

Would you recommend a friend or family member attend University of California Davis?

Recommend for what? I'm aware of the school's solid reputation in engineering and veterinary medicine, but have no notion where their other programs rank.

Recommend for whom? I have to think of an individual as to whether the programs, lifestyle, location and tuition (in-state resident vs. full) would be a fit.

Therefore, my answer to the broad question was "It depends" but that would be my answer for almost any university—not very useful for making distinctions among schools. Regrettably, it was a Yes/No question, so the interviewer was left with no response.

In your own surveys, do you have questions people would answer with "It depends..."?

There are a few common ways they'll slip in:

Trying to shorten a survey, replacing a set of targeted questions with a general one
A quest for The Metric that will encapsulate the respondent's likely action or overall impression
Relying on context—even in a survey about a specific product, "Would you recommend us?" will have some respondents thinking of your company (diverse) instead of that product line

Sometimes, all it takes is a little qualifier:

Would you recommend Tesla to someone shopping for a sports car?

Or even making the question more generic:

Overall, how would you rate our service?

In general, do you...

The more generic approach helps respondents whose "It depends" is based on inconsistent experiences.

Filed under:
Questionnaires