If You Want Better Data, Change Your Culture


If you've bought a cellphone, a motor vehicle, a home appliance, a $5 cup of coffee, or anything else resulting in an offer to take a customer satisfaction survey, chances are at some point a salesperson has requested that you respond to the survey with all perfect scores.

Most commonly, the request takes the form of emotional blackmail: "If you don't give me all 5s," or whatever a perfect score happens to be, "it counts as a 'fail,' and I'll lose my bonus" (or some other dire consequence).

Other times, it's a direct imperative: "Please rate us with all 5s." The request might be accompanied by a bribe. Customers may even face retribution for giving imperfect scores.

Sometimes, it is less direct -- a salesman showing a customer a sample survey with all perfect scores filled out, or boasting that he "always" gets all 5s.

"Allison," a professional speaker who asked me not to use her real name, discussed the surveys she once administered for a seminar company. "You show [customers] how to fill it out, and of course, you [fill] in the '10s' [the maximum score]. And then you make a joke: 'Oh, well, 10 is really average. Most people give us 20s.'"

The employee may try to place a social burden on the critical customer: "If you feel you're unable to give us a perfect score, you need to talk to us about it."

Some people may even employ subliminal methods. Allison describes how her former colleagues would do this in their presentations. They'd call attention to particular items and say things like, "Oh, this is a '10!' So you gotta write this one down."

And, of course, when all else fails, employees may simply have their friends, family, and co-workers fill out the surveys (or do it themselves) to ensure all 5s.

However, the reason for survey data manipulation is always the same: bad corporate culture.

"If you didn't get high evaluations they didn't schedule you," says Allison of her former employer. "We only get paid when we work, and if they're not giving you days to work, you're not" getting paid.

In other circumstances, an employee's paycheck -- or even the job itself -- may be genuinely on the line. One anonymous car saleswoman writes in an online forum:

I didn't want to have to ask for 5's if i didn't deserve them, but at the end of the day, i can lose a [expletive] load of $$$$$$ if i don't get them. so do you think that I am not going to ask... like hell i won't.

According to Doug Williamson, president and CEO of The Beacon Group, there has been a huge shift in corporate America in the past five years toward "facts by numbers" -- leading to stricter measurement and more accountability for managers and employees at all levels. This, in turn, results in an overreliance on surveys. Consequently, "there's no question that salespeople are being encouraged to have their customers [inflate survey] numbers."

Williamson warns against tying compensation too closely to survey results, as well as putting too much emphasis on one survey. Overreliance on one data point in a vacuum is fairly useless, even without the risk of data manipulation.

More fundamentally, he urges companies to adopt a less strict culture through better executive education and more transparency in operations.

The goal of surveying, after all, is not to get perfect scores. It's to learn.

Joe Stanganelli, Attorney & Marketer

Joe Stanganelli is founder and principal of Beacon Hill Law, a Boston-based general practice law firm.  His expertise on legal topics has been sought for several major publications, including U.S. News and World Report and Personal Real Estate Investor Magazine. 

Joe is also a communications consultant.  He has been working with social media for many years -- even in the days of local BBSs (one of which he served as Co-System Operator for), well before the term "social media" was invented.

From 2003 to 2005, Joe ran Grandpa George Productions, a New England entertainment and media production company. He has also worked as a professional actor, director, and producer.  Additionally, Joe is a produced playwright.

When he's not lawyering, marketing, or social-media-ing, Joe writes scripts, songs, and stories.

He also finds time to lose at bridge a couple of times a month.

Follow Joe on Twitter: @JoeStanganelli

Also, check out his blog .

Clean Your Data with Visualization and Algorithmic Tests

Speakers at Bio-IT World explore techniques for biotech researchers and others working with big data to identify the accurate data in their data files.

Data Sharing: A Matter of Life and Death

Cooperation among medical researchers -- done right -- very simply can mean lives saved, but the research community needs education on how to execute on that collaboration.


Re: Data collection
  • 12/12/2011 6:09:54 AM
NO RATINGS

Seth,

I agree with the analogy as well as the point.  Survey feedback needs to be done private from the subject so that the feedback remains unbiased as possible.  Asking someone directly creates a semi-dishonest response - Tell someone they did a bad job can be tedious and be a cul-de-sac when it comes to solving a problem.  It's not a great thing to say to someone's face, even if the job was done poorly. Usually the knee-jerk response is to provide an explanation and try to jiustify. Survey comments can reveal a general thought even if the comments vary to a degree.

Re: Data collection
  • 12/2/2011 10:22:46 PM
NO RATINGS

I have had sales people ask me directly to score them well, but the survey was never in person with them.  Usually, a person will ask me if they provided excellent service or ask in a less direct way. 

Either way, the person who the survey is about should never be present at the survey or able to influence in everyway.  It would be like a waiter being able to help himself to your wallet. 

Re: Data collection
  • 12/2/2011 10:22:45 PM
NO RATINGS

I have had sales people ask me directly to score them well, but the survey was never in person with them.  Usually, a person will ask me if they provided excellent service or ask in a less direct way. 

Either way, the person who the survey is about should never be present at the survey or able to influence in everyway.  It would be like a waiter being able to help himself to your wallet. 

Re: Data collection
  • 11/27/2011 9:10:29 PM
NO RATINGS

Ariella, good point. Sales quotas are one reason they might be asking you at the register which associate helped you. Could they also be analyzing how many customers require assistance and which don't in order to plan how many associates they need on a shift?

Re: Data collection
  • 11/26/2011 8:05:21 PM
NO RATINGS

@tinym I'm not very into shopping either. But, as I said, this store is on my block, and I went in after seeing the sale signs.  I did find a sweater that pretty much met my goal. Then the coats for men caught my eye because they were 50% off, and I was assured they were returnable (that was one of my questions). I bought two for my son to try one and returned one of them the next day.  The other thing they were after that day was signing people up for the store credit card. I already have a Gap card (that I haven't used in a very long time), but they still had the cashier push the Banana Republic card with an offer of 15% off that day's purchase (on top of the 40% and 50% discounts for the sale).

Re: Data collection
  • 11/26/2011 8:01:20 PM
NO RATINGS

@Broadway It's a Banana Repubic. I only started shopping there recently due to the combination of convenience (it is literally on my block) and its frequent 40% off discounts. I don't believe they pay salespeople commissions. Very few retail place do. I once asked a salesman in a shoe store that did deliver excellent service  if they received commissions. He said no because that would lead to everyone jumping on each person who enters the store. But there is some kind of tally made for a sales quota.

Re: Data collection
  • 11/25/2011 5:23:08 PM

Did you find exactly what you were looking for?  I'm not much of a shopper myself and I'm quite picky, I hope your trip went well...

Were there other questions asked of you at this retailer?

Re: Data collection
  • 11/25/2011 3:45:02 PM

Ariella, how high end of a retailer was it (if you don't mind me asking)? Some stores might want to know who helped you to simply help decide if that salesperson deserves a commission, no?

Re: Data collection
  • 11/23/2011 5:10:44 PM

Recently, I made a purchae at a clothing retailer. The cashiers must have been instructed to jot down notes because she asked if anyone helped me. In truth, no one did, except for unlocking the dressing room doors at my request. I didn't mind, though. t I'm one of those shoppers who prefer to look around myself and only ask for help if I want to know if the items are available in other sizes or colors than the ones that are out. But it seems that they want every single customer to be offered help. At some stores, they make a point of greeting anyone who enters with a hello and an offer for assistance, and perhaps they were hoping to put that into effect here. Data is gathered rather casually, but I'm sure it is being gathered. The question is, though, how to get the responses from the customers who got away -- if the cashier is the one who asks. 

Re: Data collection
  • 11/23/2011 3:13:26 PM
NO RATINGS

Maryam, if customer service staff use all their charms to get top scores on customer surveys -- but those charms involve giving great customer service and follow-up assistance if needed -- then I say the survey has done its job. It's only when "all their charms" means blatant and explicit pandering, as in, "please give me all 5s!"

Page 1 / 4   >   >>
INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +