Useful research or marketing games?
I was in my car dealership the other day, and posted behind my service representative was a copy of their satisfaction survey. Naturally I couldn't make out every line from 6 feet away, but their point came across loud and clear. On the survey, the "Outstanding" column had been highlighted in green and "PASS" marked above it. The other four levels of the scale—including one positive answer and the mid-point—had been highlighted in pink and marked "FAIL." Likewise, the Yes/No responses had been helpfully highlighted. And then of course, the form had been laminated for display.
How would you answer that survey?
As a researcher you might try to answer accurately, but even we're susceptible to that good old social desirability bias (liking and being liked).
An unhappy customer might also be accurate in their response, or the attempt at influencing their answer might even slip the marks a notch lower.
Customers who had their expectations met, who were satisfied, would probably just go for the Outstanding column. After all, the respondent has a very small vested interest in providing an accurate response, and since it seems to be life and death for the service representative, who was a nice enough sort...
If asked about the practice, I'm certain the dealer's answer would be along the lines of "Everyone does it." I recall twelve years ago getting a call from my car salesman saying I was going to be asked to complete a survey and it was very important I say everything was Excellent. I even remember his exact phrase encouraging my response: "It was excellent, wasn't it?"
So we have a huge industry collecting massive amounts of skewed data, with the front-runners prominently featuring the results in their advertising. As much as it makes me cringe as a researcher, if that's all they're doing with the data I suppose the earth will continue to spin on its axis.
I just hope they're not actually trying to use the data. While some glimmers of light may come from it, how could one possibly imagine that this method is going to produce accurate answers? And if a significant portion of people who are merely satisfied are marking "Outstanding," how can you monitor how close they are to dissatisfaction? By the time you notice a slide in how well you meet or exceed expectations, it will be because people are actively unhappy not simply neutral—at which point recovery is far more challenging.
Finally, there is the issue of how this practice influences the staff's view of customer satisfaction. Is their goal getting a mark in the Outstanding column or developing a delighted repeat customer? While sometimes these goals coincide, the first can be achieved by short-term manipulation, while the second requires consistent customer care. Even if the latter goal is the one stated publicly, the clear attempt at manipulation will generate doubts about what management really wants.
I was keen on getting all the juice out of our survey findings. Your consultation helped us do that and I would do it again!