Net Promoter Score Caveats

 |  Posted: by

Online Survey Tools

This year I have been involved in several A&U and customer satisfaction studies in which clients invariably have insisted in including the “would recommend” version of the Net Promoter Score question. I often had to argue against the tunnel vision focusing on this metric as the tell-all sign of how a business is doing.



A recent article on the subject inspired me to jot down what I have learned over the years regarding the NPS:


  • Likelihood to recommend can be driven by different factors that not necessarily have a greater impact on overall satisfaction or likelihood to use a product. I alluded to this in the article How to Link Customer Loyalty To Profits referring to the instances when someone may recommend a product based on a particular attribute, even if it is not entirely satisfied with it. This suggests that the NPS may be reflecting only aspects of a product or service that lend themselves to recommendation, which may or may not be as important for the business as a whole. A typical example of this would be a business driven by pricing. I have seen very low NPSs for retailers that people love to hate, but can’ stay away from them due to their low prices.
  • Using NPS alone can be misleading. There are product and services categories that don’t elicit a lot of enthusiasm, and some of them even suffer in general from bad reputation, but people have to use them (e.g. banks, utilities, telecoms). A low satisfaction level combined with a higher NPS compared to competitors in the category is often indicating that this may be a “best of a bad lot” case, as Randy Hanson from Marketing Strategies put it in his article Life After NPS.
  • Similar NPSs across competitors or across waves in a tracking study can mean different things depending on which size of the equation is changing. Are Promoters increasing? Are Detractors decreasing? Are Passives increasing or decreasing as a result of movement at the top and bottom? It is important to consider the changes behind the number to realize in which direction to go with remedial measures. However, looking at these changes is not enough to formulate a strategy. You need to know why the equation is changing, which requires a more in-depth driver analysis taking into account the nature of your business (product features and benefits, pricing, customer service, etc.) and the competitive landscape.
  • The Net Promoter Score is subject to volatility due to sample size. As Hanson indicates, the sample may need to be doubled or tripled to achieve the same level of margin of error previously obtained for a top 2 box score. The smaller the sample, the larger the margin of error and confidence interval in which the true value exists, which would allow for big swings in the NPS.

In short, before rushing to use NPS as an almighty business key driver, identify what is really motivating customers to come to you, and see how they impact other metrics such as overall satisfaction, likelihood to recommend and above all likelihood use your products and services. Conduct key driver analysis and triangulate all metrics in search for convergence so you don’t depend only on one metric that may not be the right for your business and that may vary wildly if you can’t afford large sample sizes.

Comments Comments

Rick Posted: September 19, 2014

I agree and disagree. 😉

Only logged in users can leave comments.