Net Promoter Score Caveats

Summary: Too many companies rely on the Net Promoter Score (NPS) to guide business decisions without realizing the limitations of this metric discussed in this article.

3 minutes to read. By author Michaela Mora on September 11, 2014
Topics: Business Strategy, Customer Experience, New Product Development, UX Research

Net Promoter Score Caveats

Talking about Net Promoter Score caveats with clients can be difficult. Using the NPS as the tell-all sign of how a business is doing leads to tunnel vision and missed opportunities. When a company gets on the NPS wagon, it is hard to get it off it.

What Is The Net Promoter Score?

The Net Promoter Score is based on the question “How likely is it that you would recommend X to a friend or colleague?

Using an 11-point rating from 0 to 10, you subtract the percentage of people who give a rating from 0 to 6 (Detractors) from the percentage of those who give a rating of 9 or 10 (Promoter). The result is the Net Promoter Score

 

Net Promoter Score Formula

 

It is a simple and intuitive metric. Above all, it is easy to calculate. However, it has serious limitations. An article by Randy Hanson, Marketing Strategies, inspired me to jot down the Net Promoter caveats I have learned over the years.

Net Promoter Score Limitations

1. Drivers of Likelihood to Recommend Maybe Very Specific

The likelihood to recommend can be driven by different factors. These may not have a greater impact on overall satisfaction or likelihood to use a product. For instance, people may recommend a product based on a particular attribute, even if they are not entirely satisfied with it. This suggests that the NPS may be reflecting only aspects of a product or service that lend themselves to a recommendation.

However, these recommendable aspects may or may not be important for the business in the long run. A typical example is when pricing is a key driver in product or retailer selection. I have seen very low NPSs for retailers that people love to hate. They can’t stay away from them due to their low prices.

2. Using The Net Promoter Score Alone Can Be Misleading

There are product and service categories that don’t elicit a lot of enthusiasm. Some of them even suffer from a generally bad reputation, but people have to use them (e.g. banks, utilities, telecoms).

A low satisfaction level combined with a high NPS, often indicates that this may be a “best of a bad lot” case, as Hanson from Marketing Strategies puts it in his article Life After NPS.

3. The NPS’ Meaning May Vary Across Competitors And Repeated Measures

Similar NPSs across competitors, or across waves in a tracking study, can mean different things. It all depends on which side of the equation is changing. Are Promoters increasing? Are Detractors decreasing? Who is moving to the Passive category?

It is important to consider the changes behind the number to identify remedial measures. At the same time, looking at these changes is not enough to formulate a strategy.

Consequently, you need to know why the equation is changing. This requires a more in-depth driver analysis that takes into account the nature of your business (product features and benefits, pricing, customer service, etc.) and the competitive landscape.

4. Sample Size Impacts The Net Promoter Score

The Net Promoter Score is subject to volatility due to sample size. As Hanson indicates, the sample may need to be doubled or tripled to achieve the same level of precision (margin of error) previously obtained for a top 2 box score for the same confidence interval.

The smaller the sample, the larger the margin of error and confidence interval in which the true value exists. This would lead to big swings in the NPS.

In Conclusion

In short, before rushing to use NPS as an almighty business key driver, be aware of the Net Promoter Score’s caveats.  Above all, research what is really motivating customers. Furthermore, test how motivators impact other metrics such as overall satisfaction, likelihood to recommend, and likelihood to use your products and services.  Therefore, you should conduct key driver analysis and triangulate all metrics in search of convergence.