The use of qualitative and quantitative research in new product development often varies depending on the field and expertise of the stakeholders in the product development process.

Consumer research to impact product development can be traced back to the origins of marketing research in the CPG industry. Traditional consumer research often combined qualitative and quantitative research. Focus groups and concept test surveys have been the dominant approaches.

With new technology and the proliferation of digital products (e.g. software, apps, websites), a call for a more agile process came at the beginning of the century. It was taking too long to get user feedback.

The ability to create quick prototypes could speed up the process and allow improvement through iteration based on continuous user research.

Unfortunately, instead of focusing on getting faster to user feedback, the focus has been moved to speeding the whole product development process while cutting costs. Ironically, user research is gets often squeezed out of this race.

High Speed, Bad Research, No Research

Quality decline, in both qualitative and quantitative research, is one of the unexpected consequences of this movement.

Designers, product managers, engineers, and developers, with little training in research, have taken over product development, especially in the digital realm. Most have a “UX/UI” label attached to their titles as if this on its own speaks to their research expertise.

Companies make it worse when they search to hire unicorns. They want people who can do everything (research, design, development).

Let’s be clear. They can’t. There are not enough hours in the day to do it all. There are fewer to do it well. Many are unaware of the training that takes to keep your biases in check when you are both the producer and the evaluator of your product. They are rarely, if ever, a good representation of their product users.

When I ask non-researchers in the UX field if they do user research, I always hear some variation of “We talk to our users.” If I dig deeper, I invariably discover informal conversations without clear direction (to be more natural) or with too much focus on specific product features (to solve the backlog).

 As “talking to users” seems like something anybody can do, the quality of one of the prime methods of qualitative research, the In-Depth Interview, has gone downhill.

Qualitative Research

Qualitative research is unstructured and exploratory in nature. This is the best approach when we don’t know what to expect when we trying to define the problem or develop an approach to the problem. Moreover, it is very useful to go deeper into issues of interest and explore nuances related to the problem at hand.

Qualitative research often uses on small samples, which by their sheer size are not representative of the target market we are trying to understand. Even if we include people with certain criteria, there is often not enough of them to be able to generalize to a larger population.

This is means, qualitative research is not the best approach for Go/No-Go decisions.

Data Collection

The most common qualitative data collection techniques are:

• In-Depth Interview / User Interview
• Focus Group (in-person, online)
• Asynchronous Online Bulletin Board
• Ethnographic observation (in-person, digital)
• Contextual Inquiry
• Diary or Journal (physical/ digital)
• Task-based Usability Interviews (moderated and unmoderated)
• Co-creation workshops

Qualitative Analysis Techniques

Qualitative research techniques tend to generate large amounts of unstructured data despite the small samples. Consequently, analyzing qualitative data to give it some structure that reveals hidden patterns in nuances is a time-consuming and arduous task.

Doing a short summary based on memory from an interview or discussion or a cursory glance at transcripts, (if any) leads often to a massive loss of rich insights that qualitative data can generate.

There are unfortunately no good text analytics tools yet that can tackle this type of data to lessen the burden of the qualitative researcher.

Use of Qualitative Research in Product Development

You should use Qualitative Research in new product development to:

• Identify the jobs the users are trying to do (JTBD), namely their needs and goals so you can develop new products that do the job.
• Explore reactions to potential perceived benefits of your product to help determine product features needed to support such benefits.
• Uncover the customer journey towards your product, including underlying motivations and factors that influence the decision to buy your and your competitors’ products
• Understand positive and negative perceptions about a product category that can affect how to position your product.
• Provide information needed to design a quantitative product testing
• Explain findings from quantitative product testing

Quantitative Research

Primary quantitative research is conclusive in its purpose as it tries to quantify the problem and understand how prevalent it is by looking for projectable results to a larger population.

This type of research uses structured data collected from a large number of representative cases, which allows for statistical analysis.

Data Collection

The most common quantitative data collection techniques are:

• Survey
• Observation (e.g. sales, visits, audits, etc.)
• Experiment (e.g. A/B Testing, test markets, etc.)
• Quantitative Remote Usability Testing
• In-Home Product Testing (often combined with surveys)

Quantitative Analysis Techniques

In new product development research, we collect data for specific analysis techniques that support new product development decisions. Depending on the objectives, we can choose one or more of the following:

• Product Concept Testing
• Positioning Concept Testing
• Conjoint Analysis
• Maximum Difference Scaling (MaxDiff)
• Pairwise Comparisons
• Market Segmentation

Use of Quantitative Research in Product Development

Quantitative Research is useful in new product development to:

• Quantify preferences for product features and product configurations to guide the product development process.
• Recommend a final course of action on which product version to launch.
• Find consensus on product appeal, benefits, and current or potential customers’ purchase intent
• Identify evidence regarding different factors relevant to usage and purchase behavior
• Test specific hypotheses about your products and guide decisions on the course of actions
• Identify and size market segments for your products
• Project results to a larger population of customers you are targeting

In Short

In conclusion, combining both approaches when developing new products, either physical or digital will give you a solid foundation to make the right decisions for your business grounded in consumer insights.

(A version of this article was originally published on February 9, 2010. The article was last updated and revised on May 29, 2020.)