Additionally, respondents may, for their own reasons, respond outright dishonestly to surveys (especially customer surveys and workplace surveys).
Even more problematically, many surveys encourage subjective, untruthful responses because of how they are written.
But with some subtle trickery, organizations can force objective honesty upon otherwise subjective respondents -- and thereby ensure higher-quality data, says Doug Williamson, president and CEO of The Beacon Group, an organizational consultancy.
- Eradicate middle ground. "The first thing you do is eliminate the fence people sit on," Williamson says. Most surveys ask respondents to rate something on a five-point scale -- or, sometimes, a 10-point scale. "Statistically... there is a disproportionate number of respondents who pick the middle of the scale," he says. "That middle of the scale, that 'three out of five,' doesn't provide the organization with any valuable data."
To keep people from selecting midpoints (e.g., threes in a five-point scale, fives in a 10-point scale, etc.), Williamson advises using a four-point scale, which has no midpoint. In his words: "You're either bad or really bad, or you're good or really good."
The Beacon Group has found that in four-point surveys, respondents are more honest, often alternating more between the three and the four and between the one and the two. Consequentially, respondents, who would be otherwise disposed to blindly giving all "perfect" scores, are more comfortable giving -- and hence more inclined to give -- lower, more objective scores.
- Sharpen definition. Theater revolutionary Konstantin Stanislavski famously said, "Generalization is the enemy of all art." It is also the enemy of analytics.
Most surveys ask respondents in a rather general way to rate someone or something with little to no specificity beyond "best" and "worst." What this does is make people feel better about haphazardly giving a rating based upon a subjective feeling.
To avoid bad data, Williamson advises, "You set a high standard on the descriptors, on the rating scale. You don't go namby-pamby." In other words, to get truly objective results, surveys must only offer answers that are so sharply defined as to impose thoughtful objectivity upon the respondent.
In one of The Beacon Group's four-point scales, Williamson says, a typical definition for a "4" might be "mastery": "The salesperson never fails to deliver masterful service." The drama of seeing wonderfully descript absolutes like "mastery," "never," and "masterful" on a survey forces a respondent to step back mentally and scale responses to the exacting language of the questions and answers.
- Use frequency as your basic metric. The two basic types of survey questions are those of frequency and those of extent. Frequency questions ask "How often?" Extent questions ask "How much?" Respondents provide more honest answers when responding to frequency questions than when responding to extent questions.
Respondents tend to "inflate" their responses when answering "How much?" questions. Conversely, Williamson explains, "When you're asked the more tangible question of 'How often did you get good service?'... you tend... to be more comfortable, and maybe even more objective."
By taking these three simple steps to ask better survey questions, organizations can reap greater objectivity and greater honesty -- and therefore dramatically improve the quality of their business intelligence.