Time to Retire the Net Promoter Score?


As much us we've been talking about the lure of the Like of late, the primary go-to customer preference metric for most marketers is still the Net Promoter Score (NPS) -- for now, at least.

Some companies are starting to turn away from or at least reduce their reliance on the NPS, the highly popular decade-old customer loyalty metric created by Bain & Company. The NPS, which predates the advent of the social web and its instant Like-ability, is culled from the responses to one basic question: "What is the likelihood that you would recommend Company X to a friend or colleague?" Respondents answer on a scale of 0-10, landing themselves in one of three categories. Promoters (10 and 9) are "loyal enthusiasts who keep buying from a company and urge their friends to do the same." Passives (8 and 7) are "satisfied but enthusiastic." Detractors (6 and lower) are unhappy customers. Bain derives the NPS by subtracting the percentage of detractors from the percentage of promoters.

Simple, right? It's too simple, Jason Faria, director of customer service at the flash online retailer Ideeli, said in a press release. Instead of relying on the NPS, Ideeli now uses the Word-of-Mouth Index (WoMI), the "next-generation NPS" launched in May by the customer experience analytics company ForeSee. "WoMI eliminates overstated detractors and has allowed us to concentrate our efforts on areas where we'll get the greatest return," Farsi said.

In its research, including 21,000 consumer surveys conducted via panel, ForeSee said it found that the NPS vastly overstates detractors. For example, detractors for the 100 largest brands in the US (as identified by Interbrand) are overstated by 299 percent on average. The WoMI measure gets around this problem by supplementing the single NPS question with one more: "How likely are you to discourage others from doing business with this company?"

This supplemental question makes all the difference, ForeSee CEO Larry Freed contends in a new report (registration required). "By asking questions that are focused on both the positive and negative word-of-mouth reviews, WoMI more accurately represents the difference between the proportions of consumers who report being highly likely to promote and those who are highly likely to detract via word of mouth."

ForeSee found the detractor overstatements by evaluating NPS metrics against its "likelihood to discourage" rating. In some cases, the NPS misstated brand detractors by 1,000 percent of more, according to ForeSee's research -- for example, there were overstatements of 1,700 percent for Heinz, 1,450 percent for Visa, and 1,050 percent for Samsung.

In the report, Freed puts this in practical terms:

Many executives use NPS as a KPI for evaluating investments and other business decisions. If an executive believes 30% of his or her customer base is made up of active detractors, but really only 10% of customers are active detractors, then the executive may very well invest significant resources into trying to "convert" detractors who simply do not exist.

This sort of situation helped swayed Nikon. "Unlike NPS, WoMI enables us to clearly understand our ROI. WoMI helps us have 'AHA' moments," Mario Castano, Internet and e-commerce technology manager at Nikon, said in the press release. The WoMI also makes information more useful, he said.

As I think about the NPS vs. WoMI, I can't help but to recollect All Analytics Community Editor Noreen Seebacher's recent post about the abysmal customer service she experienced when trying to get her Dyson vacuum fixed. In her moment of frustration, she turned from somebody who would recommend this company to somebody who publicly expressed her outrage with it. It doesn't get more discouraging than to tweet, "DYSON SUCKS!!!!!"

Now that consumers can express their opinions about companies and products so quickly and easily on the social web, I would agree with ForeSee that it's time to rethink our reliance on the overly simplistic NPS. But is the WoMI the answer? I think it's a step in the right direction, but I'm still left wondering whether there's any equation capable of addressing the social moment and effectively bottling it up in a usable metric. Your thoughts?

Beth Schultz, Editor in Chief

Beth Schultz has more than two decades of experience as an IT writer and editor.  Most recently, she brought her expertise to bear writing thought-provoking editorial and marketing materials on a variety of technology topics for leading IT publications and industry players.  Previously, she oversaw multimedia content development, writing and editing for special feature packages at Network World. In particular, she focused on advanced IT technology and its impact on business users and in so doing became a thought leader on the revolutionary changes remaking the corporate datacenter and enterprise IT architecture. Beth has a keen ability to identify business and technology trends, developing expertise through in-depth analysis and early adopter case studies. Over the years, she has earned more than a dozen national and regional editorial excellence awards for special issues from American Business Media, American Society of Business Press Editors, Folio.net, and others.

Midmarket Companies: Bring on the Big Data

The "big" in big data is no reflection of the size of the organization embracing its potential.

Push Yourself to New Analytical Discoveries

Take inspiration from Christopher Columbus as you pursue your analytical journeys.


re: Time to Retire the Net Promoter Score?
  • 7/23/2013 9:56:58 PM
NO RATINGS

Sometimes it takes baby steps... which can be OK as long as they're moving in the right direction!

re: Time to Retire the Net Promoter Score?
  • 7/23/2013 5:16:05 PM
NO RATINGS

No, I don't think they do, however, they are probably unaware of how deep one can actually go in deriving further insight.  Many organizations simply just tract their promoter scores over time and they feel that it isn't actionable.  Needless to say, I've seen it can be very actionable and we've shown several organizations how it is possible.

re: Time to Retire the Net Promoter Score?
  • 7/23/2013 4:51:34 PM
NO RATINGS

@rscollica, I agree. But, I have to ask, do you think marketers are doing enough today to dig deeper than the NPS? Do you enough of them at this point supplementing the NPS metric with deeper analytics? 

re: Time to Retire the Net Promoter Score?
  • 7/23/2013 4:38:24 PM
NO RATINGS

Yes, indeed!  Any metric that attempts to classify human understanding will arguably be incorrect at some point.  However, from what I've seen, even when customers tell you their willingness to recommend rating and then have a verbatim text, the general trends in sentiment from the text and the ratings, but certainly not exact.  So, point being, whatever metric is being used, understand the limitations, but use the analytics to the best and use this along with domain expertise to make the best possible decisions for the organization.

re: Time to Retire the Net Promoter Score?
  • 7/23/2013 4:31:35 PM
NO RATINGS

@rscollica -- thanks for joining the conversation. I think you're suggesting that NPS serves as a starting point but really should be supplemented by insight gleaned through text analytics. Did I get that right?

re: Time to Retire the Net Promoter Score?
  • 7/23/2013 3:04:37 PM
NO RATINGS

So as I read through these comments I'm struck by comments about if these scores are relevant or not.  First, I must say that if any vendor says that they can predict actual customer sentiment more than 80% correct, then they are selling you snake oil with the smoke and mirrors included!  Second, it is possible, however, to estimate the association of what they are writing about and their sentiment indicated in a promoter score.  Since people are the generators of the survey results, and also the social comments in call notes, verbatim comments/chats, and the like; any metric to measure such will be as imperfect as the people who generated the information themselves.  All that set aside, one can determine how many actual detractors there are and what they are saying can be quantified accurately enough in order to predict the scores - even if the scores aren't totally accurate.  The scores themselves will not be totally accurate due to what others have just stated in this blog thread.  The confidence in generating such estimates can in fact also be estimated (i.e. how much error or misrepresentation exists in the data?) and this too can be included in estimating how much impact an organization may have or not have by reducing the detractor topics/levels estimated in promoter score surveys.  If you'd like to know more on a methodology on how this can take place: please visit: http://blogs.sas.com/content/text-mining/ There is a lot of resources and white papers which explain the methodology much further. 

Re: Not sure
  • 7/22/2013 12:19:16 PM
NO RATINGS

Did anyone say "decline of western civilization?"

Re: Not sure
  • 7/22/2013 10:14:07 AM
NO RATINGS

Yep -- we don't want to be informed or educated. We just want to be entertained. Sort of sad, yes?

Re: Not sure
  • 7/22/2013 9:40:59 AM
NO RATINGS

Social media is winning out. Even discounting emotional responses to companies and products, it's fairly easy to guess which are valid comments. And social media is certainly more entertaining as well. And maybe that's what we're really looking for anyway.

Re: Not sure
  • 7/22/2013 3:56:08 AM
NO RATINGS

The results of ForeSee's report reveals the importance of where our attention is shifting to. The idea of asking a dual level question prevents a bias in context, that positive commentary is the only value.  Being alert to negative commentary is just as valueable, so the scoring and attention span should reflect a dual context.

Page 1 / 2   >   >>
INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +