Rule #2: Wear the respondent's shoes—it's a big payback
As much as I'm a general advocate for respondents, I know it can be a challenge when they're an amorphous group and your manager or client is an immediate voice. But, there are two very pragmatic reasons for putting yourself in the respondent's shoes long enough to make sure your survey will be a good fit.
First reason: You'll intuitively spot issues
This includes everything from incentives, to length, to instructions, to question wording, to scale structure, to missing scale options. Taking these from the detached position of a researcher—even on a careful review pass—is a very different perspective from a when you're in the respondent's shoes.
This doesn't work if you stick with the abstract demographics of your target population when you do this—it has to be more personal. I'll have a mental commentary like this to keep my role top of mind as I complete the survey, and a printout to scribble notes on:
I'm Jane Smith, a conference attendee who was sent by my employer, an Account Manager who's running ragged trying to keep up with her clients via BlackBerry while at the convention...
- What's in it for me to complete this survey? Isn't this the third one they've given me?
- How long will it take?
- There's no way I'd read that page of instructions—shorter, skimmable, bullets, distribute.
- Would I care about this section?
- Would I know what that acronym means?
- Wait, weren't they asking about that in the other section? I can't back up to look at it.
- Would I have experience with that software program? Then maybe I don't know—and that's required.
- Maybe I'd like to mark Other here or add a comment.
- How much longer? Where's the progress bar? ...
As you can see, it's not an attempt at pure method acting, but let enough of your survey manager hat go that a typical respondent's reactions have a chance to come through. And if your target population is varied, then run through with several example individuals in mind. These review passes are a "simple" trick, but surprisingly useful for locating issues in a survey and don't take long to do.
Second reason: No respondents = No data
In other words, your survey becomes an expensive, time consuming exercise to prepare an empty PowerPoint presentation. Even if you say the "No" is unrealistic and soften the statements, it's still not pretty:
- Fewer respondents = Lower statistical reliability/higher risk
- Less respondent appeal = Higher incentive costs
And while what the respondent wants isn't 100% right, just as the customer isn't always right, their instincts when it comes to flaws or discomfort are quite well developed. True, there may be times when you can't make them as comfortable as possible—the survey just has to be long, or the questions open-ended, or the issue touchy. But even at those times, isn't it better to know you've smoothed out all the avoidable bumps and made a few conscious choices rather than feel you're operating in the dark?
Ann has been a great resource for us. She responds timely, has good suggestions, and really knows her craft. Ann is our primary resource when it comes to major surveys and reports.
Barry L. Brown