Single or multiple—avoiding a common mistake

Survey designers of all experience levels make mistakes, and even when we have multiple proofreaders it's amazing what can be missed. I've seen errors ranging from typos to—more than once—overlooking an entire division of employees. These things happen.

This article is about one error that I hate to see, because it has a disastrous effect on data and is fairly easy to catch: coding a single answer scale as multiple answer or vice-versa.

When you're using standard HTML controls for your Web forms, this difference will be immediately visible as single answers appear with radio buttons or drop-down lists:

Radio button example
Drop down list example

While multiple answers appear as checkboxes (and rarely, multiple select scrollable lists):

Checkbox example

Unfortunately, two popular services, Zoomerang and Survey Monkey, use "pretty" but non-standard controls. If you're using either of those services, double-check your definition and have your testers be particularly careful to click multiple times on a scale and make sure it all behaves sensibly. I'm a fan of standardized user interfaces, so while I appreciate attractive forms, I think the best respondent experience comes from staying with conventions where they exist.

I mention testers because, just like copywriting and tech writing, we develop blind spots on our own projects. Testers don't have to be survey designers themselves, they just have to be careful and willing to spend time reviewing a survey—not just skimming questions and clicking like respondents do. Let me know if you'd like to see a testing checklist for your projects.

While we're talking about scale basics, if you're using "Other" blanks or "None of the Above" you may end up with some data cleaning. On a single answer scale, some survey software or services will allow you to submit a form which has a typed entry in the Other blank but a different radio button marked. Likewise with multiple answer scales some systems will not recognize that a mark on "None" or "Not Applicable" means no other boxes should be marked. Both of these are a challenge when you get to analysis because you either have to leave the logical inconsistency, exclude the response entirely, or try and read the respondent's mind. These types of issues are something a higher-end product like Apian's SurveyPro will have thought through.