Conjoint Analysis Myths

Summary: Conjoint analysis is a powerful technique to support new product development and pricing strategies, but it is important to understand its limitations.

3 minutes to read. By author Michaela Mora on October 22, 2013
Topics: Price Research, Analysis Techniques, Concept Testing, Conjoint Analysis, New Product Development

Conjoint Analysis Myths

Conjoint Analysis myths are spreading fast as the use of conjoint analysis methods has increased over the years in different industries.

During the 2013 Sawtooth Software Conference, Chris Chapman, from Google discussed how this family of research techniques has been successfully used to:

  • Determine feature preference
  • Predict market share
  • Find unmet needs in product portfolios
  • Determine likely response from competitors

At the same time, conjoint analysis results don’t always match reality. Clients have expectations that stem from a misunderstanding of what to expect from this research approach.

In an effort to adjust expectations, Chapman discussed the conjoint analysis myths he has encountered.

Market Share

Many users of conjoint expect to know how many people would buy a product. However, what conjoint analysis tells us is how many people prefer a product compared with other tested alternatives.

To get market share predictions we need a market-based model. This model should include not only preferences but also information about awareness, distribution, marketing, channel effects, etc. Conjoint analysis provides an important piece of the puzzle, but more information is needed.

Good Features / Bad Features 

The goal of many conjoint studies is to identify features that drive purchases. Clients sometimes interpret the results as indicators of “good” features in absolute terms.

However, conjoint analysis provides insights into the trade-offs people make among most and least preferred features.  The tested features set the context for preferences. There are no absolutes. Results can change drastically if we include or exclude features.

Pricing

 Clients often expect conjoint to tell them what the prices should be. However, conjoint is better suited to learn about price sensitivity, not to set exact price points. Results are very much dependent on the price ranges tested.

New Product Development

We often use conjoint analysis to determine what product we should develop. However, the highest average preference for a particular product configuration may hide heterogeneity in preferences. Consequently, we may end up with a product that doesn’t appeal to anyone in particular.

Preferences alone are not enough to make this type of decision. We need to consider the cost and competition. A competitor may already offer the best product configuration. Consequently, we may find better opportunities in niche areas.

In Chapman’s opinion, conjoint analysis is best to inform product lineup. We require more data and expertise for more precise optimization.

Respondent Groups

Clients sometimes perceive respondents as belonging to a particular group or type (e.g. brand preferrers) with implicit true states. However, conjoint analysis yields aggregated probabilities. Even when we estimate individual-level utilities, there are no “true types.” We should look at these groups as people with “tendencies” (e.g. they are likely to prefer brand X).

Statistical Power

Many clients prefer large samples in trying to get greater statistical power. However, Chapman advocates for using smaller samples (ideally 400).  If we use smaller samples, we may afford to replicate the study with different conjoint methods (traditional, adaptive, menu-based, etc.). The goal is to compare and validate results. This will maximize interpretative power by dividing statistical power.

Better than Instincts

Yes, the key is to learn from the conjoint data, even when we make incorrect decisions. In cases in which the data sounds counter-intuitive, we shouldn’t simply dismiss it. Often, going with the data tends to result in losses that are modest in comparison to when wrong decisions are made based solely on instincts.

These issues should not prevent you from doing conjoint analysis. However, having them in mind helps with expectation management, and leads to better use of the insights conjoint analysis provides.