What's your executive dashboard hiding?

Let's consider a set of customer service center rating questions, using a scale from 1 to 5. In real surveys these will have different means, but for illustration ours are all going to have a mean right around 3.0:

If we only use this information, we might conclude that the different factors are all rated similarly by our customers. However, if we also look at the standard deviation we get a slightly different picture:

The above chart has the mean as the centerline of each bar, and then the length of the bar indicates the standard deviation. A standard deviation tells you the range within which 68% of your respondents fall. In our example we can see that there was quite a bit of variation on all the ratings except Technical knowledge, where we have a much more tightly clustered set of responses.

So, that was a little more information—better than the mean alone—but what else might be hidden in those similar means and standard deviations? Quite a bit, as you can see when you run percentage breakdowns:

As you can see, there are significantly different profiles for each of the factors, including one where the mean represents only 1/3 the number of people as the others. In your own data, the mean and other dashboard statistics may be a useful starting point, but they cannot replace regular reviews where you drill down deeper. For this type of performance rating, a core set of statistics to review each month or quarter includes:

• Mean
• Standard deviation
• Median
• Total number of responses
• Frequency (counts or percents) for each response
• Trends over time

These will give you a fairly complete picture of what people think (without requiring a statistics book). Remember though, they still don't tell you anything about why people feel the way they do!