chart close up data 590022

Is NPS the best measure for advocacy and business growth?

Understanding and managing the word-of-mouth (WOM) behaviours of customers is critically important to organisations. According to Nielsen (2007), WOM is “the most powerful selling tool”. Recommendations are therefore highly sought after. Net Promoter Score offers a simple way for organisations to measure this behaviour. It has become extremely popular and widely used across industries. However, is this single measure really sufficient to give companies meaningful and actionable insight that will help them grow? As Grisaffe (2007) puts it “one number tells you something, but not everything”.

Background

Net Promoter was introduced in December 2003 by loyalty consultant, writer and researcher Frederick F. Reichheld. Published in the prestigious Harvard Business Review, Reichheld’s article “The one number you need to grow” argued that, instead of grappling with complex customer satisfaction surveys and metrics, companies only need one, single measure to determine loyalty and predict growth: whether a customer would recommend them. This marked the inception of the Net Promoter Score (NPS).

Reichheld (2006) distinguishes “good” and “bad” profits: “bad profits are earned at the expense of customer relationships”. Reichheld and Markey (2006) found that more than 30% of customers fall under the category of “bad profits”. NPS is designed to help organisations distinguish good and bad profits (Goldman 2011).

Reichheld’s work has been influential, with NPS being adopted by major companies including Apple, General Electric, American Express and Microsoft. His original 2003 paper has been cited 2064 times since publication, however many prominent academics remain uneasy about the claims made in it and his subsequent publications.

Claims

The claims surrounding NPS are certainly compelling:

- Reichheld states that NPS is “the best predictor of growth” (Reichheld 2006) and “the one number you need to grow” (Reichheld 2003)
- Netpromoter.com (2006) describe it as the “single most reliable indicator of a company’s ability to grow”.
- Net Promoter leaders are said to “outgrow their competitors in most industries by an average of 2.5 times” (Fry 2006).
- It is also reported that a 12-point increase in Net Promoter Scores leads to a doubling of a company’s growth rate on average (Reichheld 2006).

However, an investigation of the literature suggests a more mixed and nuanced picture than these claims suggest.

Criticisms and challenges

Criticisms and challenges to the NPS have come from a range of both academics and practitioners including: Brandt (2007), Crosby and Johnson (2007), Kristen & Westlund (2004), Morgan & Rego (2004, 2006), Nicks (2006), Grisaffe (2007), Keiningham et al. (2007, 2008), Pingitore et al. (2007), Sharp (2007), and Shaw (2008).

NPS over-simplifies a complex set of relationships
NPS has been criticised for over-simplifying a complex set of relationships. Simplicity may be appealing but as Kristensen and Eskilden (2011) highlight, it is not useful if the measure is misleading.

NPS is an outcome not a driver
NPS also does not tell us what the root cause of the problem is or what should be done (Grisaffe 2007). Freed (2006, P.5) writes: “NPS is an outcome. It is not a driver…businesses can’t directly manage outcomes. They can only manipulate the factors that influence outcomes. Customer satisfaction causes recommendation, loyalty and retention, all of which contribute to growth”.

The suitability of NPS varies depending on industry
Reichheld’s own (2003) paper identifies that NPS does not work in some industries (although he offers a blanket explanation that this is due to lack of choice in these industries). Grisaffe (2007) points out that even if loyalty can be measured using the simple, one question approach, this does not explain “the dynamics of what causally drives that loyalty” in different organisations or industries.

NPS scores are not consistent across cultures 
Cultural context may have an impact on the appropriateness of the method for scoring NPS. For instance, in a replication of Reichheld’s study in the Netherlands, Van Doorn, Leeflang and Tijs (2013) include customers scoring ‘8’ as promoters on the basis that Dutch respondents may give generally lower evaluations than American respondents. Seth et al. (2016) found that Japanese and Korean people may have a positive attitude towards the company but provide low NPS scores because they are reluctant to risk negatively impacting relationships by making recommendations.

There lacks justification for the break points
Grisaffe (2007) questions the scoring system of NPS in a more fundamental way, highlighting the fact there appears to be no justification in Reichheld’s work of the selected break points between detractors, passives, and promoters. He also questions why these “collapsed categories” would best predict customer behaviour when the increased variability of the scale points would usually be considered to provide better predictive validity. Schneider et al. (2008) point out that it is confusing that respondents scoring less than ‘7’ are classed as detractors when in fact those selecting a neutral point on the scale are not necessarily detractors.

Generational differences may also play a role
Industry research by Member Intelligence Group (2016) into credit union membership among Millennials (those aged 18-35) found that although their overall satisfaction with credit unions matches other generations, their NPS scores are lower. This merits further consideration in the light of some studies (e.g. Daymon Worldwide 2016) that suggest that Millennials may be less loyal to brands than previous generations.

Lack of empirical evidence
Researchers attempting to replicate Reichheld’s study (e.g. Morgan and Rego 2006, Keiningham et al. 2007) have not found that the NPS performs better than other metrics or has predictive validity. In fact, Kristensen and Eskilden (2011) go further, suggesting that is inefficient, unreliable and inferior to other measures such as American Customer Satisfaction Index (ASCI) or The Extended Performance Satisfaction Index (EPSI Rating; the European counterpart to ASCI).

A single measure may not be sufficient
Researchers have found that NPS is not the only customer feedback measure that correlates with financial performance (Pingitore et al. 2007, Hayes 2008) and that a combination of Voice of the Customer metrics may actually be best for predicting actual loyalty behaviours (Pollack and Alexandrov 2013, Keiningham et al. 2007). Pingitore et al. (2007) researched four net Voice of the Customer metrics – net delighted, net committed, net satisfied, and net promoter – and found that NPS was no better than the other metrics in terms of correlation to various financial outcomes.

Collecting data from a range of different measures is thought to be more reliable and less volatile than a single-item question (Hill, Roche and Allen 2007) as well as improving predictive validity (Van Doorn, Leeflang and Tijs 2013).

Furthermore, researchers at the University of Cambridge working as part of its academic-business partnership – the Cambridge Service Alliance – recommend using data mining techniques to understand behavioural components of loyalty, as well as the attitudinal components drawn from survey-based measures (Zaki et al. 2016). By looking at both attitudinal and behavioural components, they found that companies using just NPS may in fact be getting a misleading picture of customer loyalty, with a body of customers misclassified.

Research bias
Keiningham et al. (2007) point out that there was a lack of full disclosure by Reichheld and colleagues about the data used and the analysis performed which may suggest research bias.

Issues with data validity
In addition, Kristensen and Eskilden (2011) question the rudimentary nature of his statistical methods as well as the lack of scientific rigour indicated by his response to this concern in which he argues that he sees “little value in continued debate about cause versus correlation, timeframes, or statistical methods”. Grisaffe (2007) points out the problematic nature of Reichheld’s (2003) use of longitudinal growth data but only cross-sectional NPS data in his data, also noting that some of the growth data was collected before NPS was measured, weakening his claim of NPS’s predictive ability.

 

References

Grisaffe, D., 2007. Questions About the Ultimate Questions: Conceptual Considerations in Evaluating Reichheld’s Net Promoter Score (NPS). Journal of Consumer Satisfaction, Dissatisfaction and Complaining Behavior, 20, 36-53.

Hayes, 2008, "The True Test of Loyalty," Quality Progress, June 2008, 20–26.

Hill, N., Roche, G. and Allen, R., 2007. Customer satisfaction. London: Cogent, p.226.

Keiningham, T. L., Cooil, B., Andreassen, T.W., & Aksoy, L. (2007). A longitudinal examination of net promoter and firm revenue growth. Journal of Marketing, 71 (July), 39-51.

Kristensen, K. and Eskildsen, J., 2011, September. Is the net promoter score a reliable performance measure?. In Quality and Reliability (ICQR), 2011 IEEE International Conference on (pp. 249-253). IEEE.

Leisen Pollack, B. and Alexandrov, A., 2013. Nomological validity of the Net Promoter Index question. Journal of Services Marketing, 27(2), pp.118-129.

Member Intelligence Group, 2016. Millenial Research Survey. http://www.memberintelligencegroup.com/wp-content/uploads/2016/08/MIG-Millennial-Research-Survey-Report-8.2016.pdf

Morgan, N.A. ÔC Rego, L.L., 2006, The value of different customer satisfaction and loyalty metrics in predicting business performance. Marketing Science, 25, 5, pp. 426-439.

Pingitore, G., Morgan, N.A., Rego, L.L., Gigliotti, A. & Meyers, J. (2007) The single-question trap. Marketing Research, 19, 2, pp. 9-13.


Reichheld, F.F., 2003. The one number you need to grow. Harvard business review, 81(12), pp.46-55.

Schneider, Daniel; Berent, Matt; Thomas, Randall; Krosnick, Jon, June 2008), "Measuring Customer Satisfaction and Loyalty: Improving the 'Net-Promoter' Score" (PDF). van Haaften. Berlin, Germany: Annual Conference of the World Association for Public Opinion Research (WAPOR).

Seth, S., Scott, D., Svihel, C. and Murphy-Shigematsu, S., 2016. Solving the Mystery of Consistent Negative/Low Net Promoter Score (NPS) in Cross-Cultural Marketing Research. ASIA MARKETING JOURNAL, 17(4), pp.43-61.

Van Doorn, J., Leeflang, P.S. and Tijs, M., 2013. Satisfaction as a predictor of future performance: A replication. International journal of research in marketing, 30(3), pp.314-318.

Zaki, M., Kandeil, D., Neely, A. and McColl-Kennedy, J., 2016. The Fallacy of the Net Promoter Score: Customer Loyalty Predictive Model. Cambridge Service Alliance.

This entry was posted in Market research, tagged Membership, Market Research and posted on September 28, 2018


<< Back