Late last year “What’s Wrong With the Net Promoter Score” by Augustine Fou was posted on ClickZ. At the time I flagged it for detailed review and response because it contained a large number errors and mis-statements that I wanted to comment upon. As it happens, when I went back to review the article I found lots of other people felt the same, judging by the volume of comments attached to the article.
Before I go further I should say that I’m not a blind supporter of NPS. I only support it because it has been shown empirically to work. I support it for the same reason that I use the telephone: I don’t know exactly how it works but it does, and I’ll continue to use it until it stops working or something better comes along.
Before we dive in you can check out Wikipedia for a quick refresher on Net Promoter Score.
(Update 30 Apr 2010: Or you can download our free handy dandy Introduction to Net Promoter Score.)
There are so many mis-statements in Fou’s article that I could go though it line by line but that would be a little tedious so I’ll break it down into the same three summary points provided by Fou:
“NPS doesn’t tell me anything new”. This is patently false.
Perhaps the key feature of NPS is that for the first time it tells us the most important thing: whether a customer is feeling loyal and how loyal all of our customers feel. For the last 50 years companies have been trying to get a handle on exactly this question. We’ve spent countless hours and millions (perhaps billions) of surveys to capture this one piece of information. For years we tried measuring “customer satisfaction” but that has been show to be poorly related (and poorly correlated) to customer loyalty.
It also tells us this, in a simple to understand, practical, predictive way. If we increase NPS then our revenue will probably rise, if NPS goes down then revenue will probably drop. This cannot be said for any other single measure that I am aware of.
Lastly, all products lend themselves to word of mouth, but the percentage of the population interested varies dramatically by category. NPS does not stand alone for all markets, true, but customers NEVER judge you in isolation, they always see you and your brand in the context of others. In fact the best comparison is to your own last NPS score, which brings us to the next point.
“NPS is based on flawed math”. This is empirically false.
Fou states that NPS is based on a “seemingly arbitrary 11 point scale”. At which point I would suggest that all scales are arbitrary (except binary?). NPS is no more arbitrary than 1-10, 1-7, etc used for other surveys.
However, reliability and validity are the only criteria that are relevant in this discussion. It is not the scale that is used but the fact that it works that is important. The analysis behind the survey indicates that this scale, regardless of it’s theoretical underpinning, works and that’s enough for me.
With thanks to Satmetrix
The article also points to NPS being attitudinal rather than behavioural, i.e. “How many times did you recommend company X”. For my part I have always consider that it’s not actually about whether you HAVE recommended a company, it is asking WOULD you lend your personal credibility to this company. So it really has little to do with the actual event of recommending a company as it has everything to do with how you feel about recommending a company.
“Not actionable”. This is only true if you let it be.
Sure if all you ever did was run an NPS survey, take the number and report it, you do not have an actionable approach. Incidentally, this is what occurs in a very great number of customer satisfaction surveys, “likeliness to buy” scores and countless other approaches. The company gets the report and looks around at each other asking: “So what now” or “So does anybody know why it went up?”
Nobody should be naively using NPS without collecting additional information to allow a root cause analysis of what is different between promoters and detractors. In fact, if your customer segmentation does not systematically explain the difference between promoters and detractors it is probably not very useful for marketing and definitely not for CEM.
Using NPS as the key metric of loyalty, you can create closed-loop processes so that the right employee directly investigates the root causes that drive customer response to the NPS question. Then you are acting directly on the drivers of customer loyalty.
NPS is a cross departmental strategy to focus the organisation on how it is treating and impressing customers, not a standalone ‘what is the meaning of the universe’ answer. This is a business indicator that has the customer in the centre and takes us back to Deming and his approach to building strong customer focused businesses.
At its heart this is why NPS is so useful: because it provides us with a clear proxy for customer loyalty that we can act on in a purposeful and methodically manner.
Finally, Fou’s article closes with a suggestion that a better approach than NPS is to use changes in search volume to gauge success. Suggesting that search volume is a good indicator of customer loyalty seems a little light-weight. This approach appears to be more of a way to fine tune the sales and marketing information process than understand the drivers of customer loyalty.
Net Promoter, Net Promoter Score and NPS are registered trademarks of Bain & Company, Inc., Satmetrix Systems, Inc., and Fred Reichheld
No related posts.
Thanks Adam for your response to my article on ClickZ – http://www.clickz.com/3635696
I’d like to cite that a lot of other people have done research that shows
that the NPS has little predictive value and pretty much states the obvious.
- specifically, if your product or service is awesome, people will recommend
- if people will recommend, your sales will go up
- the NPS summarizes attitude and not behavior (i.e. when someone will make the recommendation and therefore whether you can predict sales increases based on it.
See this reference:
http://blog.vovici.com/blog/bid/18204/Net-Promoter-Score-NPS-Criticisms-and-Best-Practices
Augustine,
At the risk of getting into a war of words, I don’t deny that there are critics of NPS and others that have tried and failed to exactly repeat the work done by Satmetrix. Nor do I deny that NPS pretty much states the obvious. The importance from my perspective is that the NPS score has a good (or useful) correlation with future customer behaviour and thus an easily collected proxy for loyalty. This is the critical point of difference to, say, “customer satisfaction” which lacks correlation and for which you could make the same case of “stating the obvious”.
Finding a proxy for customer loyalty is the first step in creating a system to build and improve loyalty. If you can determine which customers are feeling more and less loyal then you can determine how they are different and why they feel that way. Then you can act on those areas to improve your business.
Critically, NPS is not an end in itself; it is but a means to an end.
(PS — I know that correlation does not automatically mean causation but it’s a better starting point than no correlation.)
What does not seem clear to me is how you would define “feeling loyal”?
I believe that it is a very important question to understand what is meant by “feeling loyal to”
If a person would happily recommend a product/brand, does it necessarily mean that this person would answer “yes” to a question like “Do you feel loyal to brand X?”
Thank you for your comment
Gwarlann
Gwarlann,
I agree that you can’t ask a question like “do you feel loyal” with a yes/no response and expect to get a useful answer. I don’t like the use of the term “feeling loyal” in any case because it can be so many things to so many people. Instead we use the concept of “empirical loyalty”, i.e. loyalty proven by spending patterns and repurchase. If someone re-purchases your product they are empirically loyal. Where I use the term loyalty I mean this empirically loyalty not emotional loyalty.
The Net Promoter Scores process appears to be able to predict which purchasers will be empirically loyalty even if cannot encapsulate a neat value on emotional loyalty and so it is useful in helping to shape your customer management approach.
Adam
Thank your interest and comments on my blog post: 7 Metrics for measuring “loyalty” that counts on our blog Results Count … everything else is conversation.
http://tiny.cc/j33gc
I would generally agree with your comment that we are in agreement on “do what matters”.
I have no qualms about the utility and use of NPS as one potential indicator of customer satisfaction. It is elegantly simple in design and execution.
While a number of studies have shown that NPS can correlate with company growth, far too many companies fall into the trap of just making the assumption that a) NPS will correlate to their growth, and b) correlation equals causation.
NPS is a potentially vaulable loyalty predictor, but companies need to balance the scorecard with actual results outcomes in terms of sales generated by satisfied customers.
In short, NPS is one loyalty metric … we can NOT always assume that it accurately predicts, especially if the sampling and execution are poorly executed. We also need to balance the scorecard with outcome metrics, which in turn enable intra company validation.
Thanks for the dialog … and useful background information on NPS and how to make it more valuable as a predictor metric.
Chris Petersen, PhD
CEO Integrated Marketing Solutions
Chris,
Yep I think we in the same space here but we maybe disagree on how.
Every metric can be poorly implemented, including NPS and traditional sales reporting; do we include or exclude bad debt, how bad, etc? So we must not discount an approach just because it can be poorly done.
Incidentally, in general, I favour a traditional (can we use that term yet) balanced scorecard of measures, rather than just one.
The real issue with any metric approach is turning feedback into action. Measuring for measuring’s sake is worthless.
The best NPS implementations do not use NPS as an end in itself. The NPS process is a diagnostic to find the areas and touch-points in the business that are letting down the side. You then extend the NPS process with additional quant and qual feedback to understand how to fix the business and drive up value.
Adam Ramshaw
Adam,
I think that you “nailed it” in your reply. The key is truly a balanced score card of measures that includes NPS as well as results outcomes.
In this economy, far too many US companies have rushed to NPS and focus on it as the exclusive measure of satisfaction and loyalty. While they may have simplified and cut costs, a single NPS metric is a very dangerous strategy.
My PhD is in statistics and measurement, and I am very interested in finding more research with analyses of co-variance that quantify the predictive value of NPS for results outcomes like sales, mix, market basket. I would be very interested in learning more … or having you point me to some qualtiy research with careful design, sampling and stat.
I’m starting to look for guest bloggers. I like your style and the quality of your blog very much. If you have an interest in being a guest blogger on Result Count, please let me know.
Chris Petersen
Chris,
As I said, I thought we were in the same space and now it’s clear that we are. Silver bullets don’t exist in life or in business and a range of approaches is always needed.
With regards to additional well designed and published research on NPS, other than that directly related to Satmetrix or Bain, it is a little hard to come by; although some does exist. This paper, “Advocacy Drives Growth”, London School Economics, December 2005 provides some additional independent validation. http://www.ianbrooks.com/useful-ideas/articles_whitepapers/Advocacy%20Drives%20Growth.pdf
I’d be happy to guest blog over at Result Count.
Adam Ramshaw