Quick — what’s the worst customer satisfaction questionnaire you’ve ever seen? They’re ubiquitous — absolutely everywhere. While I guess the intent to gain customer feedback should be applauded, I feel little sympathy for those who fail because they didn’t bother to proofread or pre-test a questionnaire. There’s really no excuse for the basic mistakes we see every day.
Here’s a recent example of a questionnaire with a problem.
During a concept test of a new restaurant’s menu, a survey form was prepared that asked participants to rate each of the three salads they had sampled on a number
of important criteria. How was the size, the overall appearance… the taste, value and so on? For each criteria, and for each salad, the form included a space to record a rating. Fifteen spaces in all.
Once they were finished rating all three salads, the customer satisfaction questionnaire asked participants
“Would you buy the salad? Yes / No”
Apparently expecting a ‘Yes‘ or a ‘No‘ for each salad, the questionnaire authors got nothing. Fewer than 25% of the participants answered the question. For those that did, there was no way to use the data collected.
The authors simply overlooked the fact that they had included only a single space for the response – not three – as would be needed to rate each salad.
Which salad were respondents being asked about? Glancing at the form, there was no way to tell. For those that tried to answer, analysts had no way to discern which salad was being referred to. Or if the answer was just an overall response…
‘Yes, I guess I’d buy at least one of these’.
It’s not that a making such a mistake is unforgivable. We all make mistakes. It’s that this mistake would have been caught before it got to the field if the questionnaire had been pre-tested with someone not involved in its development. It takes a fresh set of eyes to notice things like this.
Which brings me to the point. Why will researchers go to all of the expense of putting together a customer satisfaction questionnaire and to entice customers to provide their feedback, without taking an extra few minutes to make sure they’re going to collect meaningful data?
In the case of the three salads, analysts were left with no data for arguably the most important question on the topic. Given the cost of conducting multiple focus groups that included prepared food samples, the exclusion of a few blank lines turned out to be a very costly – and completely avoidable – mistake.
The lesson in all of this?
Make sure someone pre-tests your questionnaire – even if you’re sure it’s ready to go. You’ll be glad you did!