Comments
Why One-Question Surveys Make for Bad Analytics
View Comments: Newest First | Oldest First | Threaded View
Page 1 / 4   >   >>
Re: Survey frustration
  • 10/4/2011 2:16:13 PM
NO RATINGS

I'm wondering if it is truly a one question survey. If a person presses 4 for being dissatisfied are they emailed a follow up survey to get more information? 

The reason I wonder is that Netflix is consistently rated one of the highest web  in customer satisifaction.  This year they are only one point behind Amazon. 

Re: Survey frustration
  • 9/24/2011 6:27:12 PM
NO RATINGS

@Cordell  Excellent point regarding Banks, an industry fraught with "agnostic arrogance" regarding customer service.  And you are so right, banks are missing the point, people are not changing banks as often due to this disregard for their time and business, as they have little choice.  So what is analytics to make of this ?  Not much unless you measure the right variables and come from a place of geniune concern for the customer.

Cordell's point about customer service = cost
  • 9/24/2011 3:52:31 AM
NO RATINGS

When I read about the customer service, I felt a similar vibe to Cordell's point about cost and customer service.  He's right but it's not just banks. Many businesses consider customer service as strictly a cost rather than a chance to change the value of the operation.  That's essential what Zappos proved with its customer service (and with its hiring practice of paying prospective employees to quit).  Now that there's been a succesful model, more consideration of how to improve customer service will come into practice.  The only question left is which companies will continue to refine and succeed.

Re: Survey frustration
  • 8/26/2011 4:47:47 PM
NO RATINGS

That's an excellent point about banks and loyalty.

Re: Survey frustration
  • 8/26/2011 2:41:13 PM
NO RATINGS

An anecdote about phone trees. I was reading an article complaining about the elaborate phone trees your bank requires you to navigate when you call that included this gem - "It's almost like they'd rather you didn't call at all"  Um, that's exactly what it is.  For a bank calls=cost.  If the CSR can't cross-sell something when they get you on the line then they'd rather not take the call at all when a machine can do it.  Shortsighted? Maybe but seriously, when was the last time you switched banks?  What some banks may interpret as loyalty is nothing more than being averse to switching!

Netflix might do the same, mistaking loyalty for a lack of alternatives.

Re: Thinking to deep
  • 8/26/2011 2:28:03 PM
NO RATINGS

Okay let me turn this scenario on it's head.  I'm speculating of course but what if the survey isn't really designed to evaluate specific customer feedback per se.  Instead it's a tool to evaluate the CSR's. (Beth a good question when/if you get the interview!). This would lead to all kinds of unusual behavior like reps disconnecting before you could respond to a bad interaction or encouraging you to call later to push you off on another rep.

Now let's really get crazy and imagine that someone at Netflix decided that this was useful market data and upon observing that that most customers are "satisfied" (because the data's been skewed) they conclude that they must be less price sensitive - hence the change in pricing structure.  Well you can't say they didn't take action!

Bad decision based on bad data?  I'm positive I'm overthinking now. Surely Netflix realizes that their loyal customer base isn't going anywhere based on more than thier phone surveys.  The lesson here is that not thinking through design of the survey and the process for data gathering can bring unintended consequences.

Re: Thinking to deep
  • 8/26/2011 1:08:50 PM
NO RATINGS

Joe,

True. It's all about action from a service P.O.V. If you can't act decisively based on feedback then, in essence, you have nothing. SO much like standard marketing with calls to action, surveys must provide the same answers to those who are looking for answers.

Re: Thinking to deep
  • 8/26/2011 11:44:39 AM
NO RATINGS

I haven't taken one of the customer support surveys, so I don't know what if any other options are available after selecting a choice.

It there aren't any, you get high-level overview of what your customers think (as I mentioned not very useful in its own, but gives you the temperature). What they should do is break this down a few more steps afterwards.

This sounds to me like they're getting something started and are taking the approach very slowly and simplistically. Too much change for a user is scary.

If this is all they have planned for the future, than I agree, the data isn't going to be worth anything long term. 

Re: Thinking to deep
  • 8/26/2011 10:18:34 AM
NO RATINGS

But scucci, the knowledge as to whether I am "satisfied" in some vague, generic way is not actionable.  It does not tell you why I feel that way, what I feel that way about, or what you can do to maintain or change that feeling.

It's meaningless, and can only lead to blind guesswork.

At least the e-mail surveys asking about picture and audio quality, while they do have their weaknesses (as an earlier commenter pointed out), tell you *something.*  So to with the e-mail surveys asking when a particular DVD arrived or was shipped.

But "yes or no: are you satisfied" with no in-between or explanation (either of what the question really means or what the answers really mean)?  You may be able to impress a clueless senior executive or board member with a measurement of the numbers on that metric, but it doesn't tell you anything productive.  Strictly from the perspective of gathering and applying analytics, you may as well abandon the question and just look at sales volume and costumer service call volume.

Re: Need Marketing researcher
  • 8/26/2011 10:12:38 AM
NO RATINGS

First let me state that I don't work for Netflix (LOL) and that I might be the only person on the this and other boards that like the UI.

What we have to remember is that we're used to having the ability to customize our systems when we don't like an option.

What Netflix is doing with their UI is making it as EASY to use as possible for the AVERAGE user. When I say average user I'm thinking about my son and grandmother. They're not concerned that it takes slightly longer or doesn't have the options that we want, because they want to streamline everything to make it easy to use.

This gives better customer satisfaction since they know how to use it without reading directions or calling customer support.

I think market research is right on the money if you're an average user.

Page 1 / 4   >   >>


INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +