I can’t emphasize enough the need for realism in price research in order to get valid results. We are well aware that in the context of a survey, respondents are likely to over- or underestimate their willingness to pay for products and services (more pronounced in certain categories than in others), so we need to strive for research designs that minimize the gap between the survey context and actual purchase behavior.

An interesting study comparing different approaches to price research, with realism in mind, was recently published in the 2011 February issue of the Journal of Marketing Research (How Should Consumers’ Willingness To Pay Be Measured? An Empirical Comparison of State-of-the-Art Approaches by Miller, Hofstetter, Khromer and Zhang).

In the article the authors present a comparative study of direct and indirect methods to measure willingness to pay to actual purchase behavior of a cleaning product. The methods examined include:

1. OE: Direct questions asking consumers to state their willingness to pay using an open-ended question format (Direct)
2. CBC: Choice-Based Conjoint Analysis, which calculates willingness to pay based on consumers’ choices among several product alternatives and a “none” choice option (Indirect)
3. BDM: The Becker, DeGroot and Marschak approach, in which the participants are obligated to purchase a product if the price drawn from a lottery is less than or equal to his or her stated willingness to pay in response to a direct question (Direct)
4. ICBC: Incentive-Aligned Choice-Based Conjoint analysis, in which participants are also obligated to purchase a product if the price drawn from a lottery is less than or equal to the  inferred willingness to pay based on his or her choices among several product alternatives and a “none” choice option (Indirect)
5. REAL: in which participants were asked whether they would buy a product using a dichotomous format (Yes/No) at a certain price displayed in an online shop similar to that of the product manufacturer. This data was treated as real purchase behavior. The prices shown in the online shop corresponded to the price levels used in the conjoint analysis groups (CBC and ICBC).

All participants who were obligated to buy in the BDM and ICBC groups, or agreed to buy in the REAL group, were sent the product and an invoice by mail, which they could pay in cash or by credit card.

The most significant finding of this exercise is that in terms of the absolute distances between an elicited willingness to pay and the actual purchase behavior, the BDM group showed the least deviation from the REAL group, followed by ICBC, OE and CBC. In other words, the BDM and the ICBC approaches elicited a more realistic response to different price points and yield results closer to those of the REAL group, which is not surprising as these approaches put participants in an actual purchase situation.

 However, I was surprised to learn that OE did better than CBC. These results stay in contrast with other studies (Ding, Grewal, Liechty, 2005) and the results from a second dataset the same authors analyzed (digital compact camera) where CBC performed better than OE. The authors hypothesize that CBC is likely to perform better when a product is less unique and faces more competing products, which was not the case of the cleaning product tested. This makes sense and tells us that we should consider the product category when deciding which willingness to pay measurement approach we should use.

 As an overall conclusion, the authors suggest that market researchers should consider incentive-aligned approaches such as BDM or ICBC when:

• A product prototype exists
• The underlying product is not expensive
• The cost of fulfilling the actual buying obligation is low

Although, the study itself has several limitations (assumptions of market monopoly, small number of CBC tasks, limited to one product category, low completion rates and relatively low understanding of the task under incentive-aligned conditions), it gives support to the idea that we are likely to get more realistic results in price research if we provide test conditions that are as close as possible to the actual purchase situation.