Getting to a State of 'Quantified Confidence'

I seemed to have touched a chord last week when I wrote a blog post asking whether it was time to retire the Net Promoter Score, or NPS.

All Analytics community members generally agree that the NPS, a much-used customer preference metric, as a measure unto itself is of increasingly limited value. What most seem to be unsure of, however, is what to do about that. One commenter, for example, wrote that the NPS has indeed "gone the way of the dodo." However, he continued by saying he was not sure what the best replacement might be. "Things happen quickly in social media, and many are influenced by what happens to their friends and what they have to say. I think these factors make it really hard to predict behavior. I think overall loyalty is decreasing across the board."

Perhaps, then, the discussion ought not to be about retiring the NPS, but of supplementing it, as well as survey results, with more advanced analytics. Randy Collica, a senior solutions architect supporting the communications, entertainment/media, and travel industries at SAS joined the conversation with this suggestion: "… whatever metric is being used, understand the limitations, but use the analytics to the best and use this along with domain expertise to make the best possible decisions for the organization."

In a follow-on email with All Analytics, he stressed the importance of moving beyond just the NPS. "There really is so much that organizations can do than just trend the past, and we're attempting to move them in the right direction."

Painting the big customer picture is best done by combining survey results with a promoter score and text analytics, along with predictive modeling. In doing so, companies can gain a much better understanding of what their customers think about their products and services -- and of the monetary impact of those attitudes. This is the crux of the matter, after all. As one community member commented, "The real question is how does the score translate into sales?"

For more on how this would work, Collica pointed to the SAS whitepaper, "From Customer Risk to Corporate Strategy: Using Text Analysis and Predictive Modeling to Improve Promoter Scores." The whitepaper suggests:

    To quantify the impacts to the entire customer population, organizations can extend their survey results by using the promoter scores to predict the behavior of customers who were not surveyed. One method of doing this is to gather all of the customer contact history and combine this unstructured text with the survey promoter score responses -- as well as with the structured data the organization already has. Unstructured text can include verbatim comments from the promoter survey, call center notes, and comments from sales or technical support representatives.

The analysis can actually predict promoter scores, which when combined with customer revenues and other financial metrics, the whitepaper says, can show the relationship of the likelihood to be a promoter (or detractor) with potential revenues. "Then you can use this relationship to develop strategic plans centered on the voice-of-the-customer (VoC). For example: If X customers are moved from passive to promoter status, what is the effect on overall revenue?"

The idea is to perform simulations on data using econometric modeling methods to glean insight into how much value the company can achieve by moving a customer up the promoter score curve. Below you'll see an outline of the methodology. The key, the whitepaper explains, is to turn the information contained in customer call notes, transcripts, and support staff logs or emails, those types of unstructured data sources typically helpful in predicting customer sentiment and likelihood to purchase or recommend, "into something that is structured so that predictive models can be developed and applied to the entire customer population."

Overall promoter-score analytics flow (source: SAS)
Overall promoter-score analytics flow (source: SAS)

I won't go until full detail here, but I do recommend that anybody looking into how to boost the effectiveness of decision-making around NPS dig into the whitepaper. What you'll learn is how to better use the promoter score and survey results to figure out the value in migrating customers up the promoter score curve. And that, in turn, will lead you to a state of "quantified confidence."

Can you see the value in mining text and using predictive modeling to gain more confidence in your customer-related decisions?

Beth Schultz, Editor in Chief

Beth Schultz has more than two decades of experience as an IT writer and editor.  Most recently, she brought her expertise to bear writing thought-provoking editorial and marketing materials on a variety of technology topics for leading IT publications and industry players.  Previously, she oversaw multimedia content development, writing and editing for special feature packages at Network World. In particular, she focused on advanced IT technology and its impact on business users and in so doing became a thought leader on the revolutionary changes remaking the corporate datacenter and enterprise IT architecture. Beth has a keen ability to identify business and technology trends, developing expertise through in-depth analysis and early adopter case studies. Over the years, she has earned more than a dozen national and regional editorial excellence awards for special issues from American Business Media, American Society of Business Press Editors,, and others.

Midmarket Companies: Bring on the Big Data

The "big" in big data is no reflection of the size of the organization embracing its potential.

Push Yourself to New Analytical Discoveries

Take inspiration from Christopher Columbus as you pursue your analytical journeys.

Re: Actions over numbers
  • 7/29/2013 7:47:58 AM

Right -- the metric shouldn't dictate the metric, the metric should help assess the the results of the action! Good point.

Re: fuzzy
  • 7/29/2013 7:46:06 AM

Hi Jeff, that's sounds like a reasonable idea if any! Try it out for your organization/project -- and decide if it works or not. What works for some won't for others, after all!

Re: Agreed
  • 7/29/2013 7:44:13 AM

Hi Pierre, so are you suggesting we keep with NPS and supplement or that we use NPS as the basis to develop a metric that takes into account greater complexity? 

Re: Agreed
  • 7/28/2013 10:17:00 PM

I think it is certainly apropo, Beth. The value of metrics needs to be examined periodically, but maybe a complete revamp of an industry standard is not necessary. We have so much capability for developing a compound metric.  I am not suggesting reporting for reporting sake, but this illustrates how difficult changing an adopted metric can be.

  • 7/28/2013 8:45:45 PM

I read the original post on this NPS.  I'm not in marketing so it's all new to me too.  The only thing I kept coming back too was that it seemed fuzzy.  Just not very mathematical or precise at all.  This seems to help.  But still.  I guess, for me, I would need to do an NPS project to really get it. 

Re: Agreed
  • 7/28/2013 1:48:46 AM

Bulk, I agree with you. NPS can help marketers to assess the financial impact of their actions provided the NPS is linked to the financial model that quantifies everything in financial terms.

Actions over numbers
  • 7/28/2013 1:42:19 AM

Developing a NPS and then deciding the actions is not the right way to go. Firstly action plans to build customer loyalty must be outlined and then trying to derive what impact each action has on customer loyalty on the NPS is the way to go. Often vision is lost if planning is started by keeping in view targetted numbers; instead the targetted market position should be thought out and then impact of actions to reach there be quantified.

Re: Agreed
  • 7/26/2013 8:12:26 AM

Tom, I'd never heard that Chinese proverb but it does seem awfully apropos to this situation (as well as many others in the world of business and technology adoption).


Re: Agreed
  • 7/26/2013 2:02:40 AM

I am inclided to agree as well, given the intergration and expertise surronding the system I think it makes far more sense to improve it than replace it. 

  • 7/25/2013 8:58:04 PM

There is a lot of experience and know ho around NPS and for that reason alone it makes more sense to improve it than replace it all at once. I am sure we will develop better measures, but an old Chinese proverb says that the way to cook a frog is to put him in a pot of lukewarm water and slowly heat it up until he is cooked. If you throw him into boiling water he will jump out.