Using survey respondent panels

So what's a survey panel?

It's a database of people who have signed up to complete surveys for compensation, whether it's accumulating points or cash payments. As part of the sign-up, they provide detailed demographic information, as well as information on hobbies that allow the panel vendor to sub-set the list.

While these could be more general lists than a niche magazine's subscribers, panels have several advantages:

• Participants are willing to take surveys
• Payment mechanics are taken care of, including "micro payments" which might be expensive to administer yourself
• Panel vendors sell you qualified completions—respondents, not leads
• Because the panel service handles all respondent contact, you don't have to identify your firm as the project sponsor
• Most panels are e-mail driven for fast survey turnaround (don't believe 99.999% of the people selling "opt-in" e-mail lists)
• The panel vendor can provide or recommend compatible Web survey hosting services

While there is a quality concern about panels being overworked or loaded with "professional" respondents, in general panels are not viewed as better or worse than other lists. The main downside, as I'm sure you've already guessed, is cost. Because the panel vendor is selling you 500, 1,000 or however many completions you need, they're taking a gamble on how much they'll have to pay respondents to get responses. The two factors that drive the cost are the survey length and incidence rate.

For the survey length, use skips as much as possible to tailor the surveys to individuals. 5 minutes shaved from the completion time will reduce the panel cost significantly.

The incidence rate is how many people from your sample will qualify to complete the survey. This will vary from vendor to vendor for the same survey, since different panels will have different demographics/hobbies that they're using to match the sample to your population. For example, if you're looking at urban bar-hopping habits, you'd start with a sample filtered by region and age. Assuming the panel vendor didn't have nightlife habits on file, you'd need to begin the survey with qualifying questions about how often respondents go out. If 20% of the people pass your screening questions for the survey, that's your incidence in the sample. Lower incidence = higher cost.

So, if you have a 5 question survey about laundry detergent, you're going to have a relatively inexpensive panel. If you've got a 150 question B2B study, be prepared to invest in your data. To give you a couple examples, here are two projects I've had quoted:

Technical requirements

Your survey is going to need to handshake with your panel vendor, which is typically done by embedding information in the URLs that begin and end the survey:

• Receiving the member ID as a password to start the survey

• Exiting an unqualified respondent (doesn't bar hop enough)

• Exiting a respondent who exceeds a quota you've set (21-25 year olds)

• Exiting a completed respondent for compensation

These are not functions built into low cost survey tools, and getting different systems to speak to each other can be a challenge—part of why panel vendors also offer hosting. If you're not using an integrated solution, be very careful to understand how your survey cut-offs are managed, so you don't end up with excess respondents.

Finding a vendor

Three established survey panel vendors are:

• Survey Sampling http://surveysampling.com/
• Greenfield Online http://www.greenfield.com/
• GMI http://www.gmi-mr.com/

I tend to approach vendors with what a very specific definition of who I want, but leave it up to them to get creative with segmenting their database to match. That's part of what was behind the range in those quotes—some vendors were better than others at increasing the incidence rate in the sample.