Why Validation of Qualitative Research Is Needed

Validation of qualitative research insights is a must to support important business decisions.

In my many years in the market research insights industry, the most common obstacles to qualitative insights validation I have seen include:

• Lack of understanding of how different quantitative approaches may validate some of the insights found through qualitative research from either higher decision-makers or qualitative researchers leading research projects.
• Lack of budget, which often is connected to the reason above. If decision-makers don’t understand the limitations of qualitative research or qualitative researchers are limited in their knowledge of quantitative methods, then the efforts to find the money to do it are weak or nonexistent.

Different Skills Needed

Based on personality and preferences, many market researchers find themselves favoring qualitative research or quantitative research.

Understandably, they require different sets of skills and knowledge base, and switching between the two is a hard task. I can attest to that. I have been doing it for many years. Difficult, but oh so rewarding!

Caveats

In the small samples that are typical in qualitative research, sometimes we can miss the underlying themes behind all the in-depth details we find so fascinating. At other times, we can uncover themes, even from small samples, but we can’t be certain they are representative of the larger target population we are studying. We may be staring directly at the consequences of sample size saturation. 

The idea of sample size saturation is simple. We add new participants (IDIs or focus groups) until no new information on the topic of interest is uncovered.

The problem is that sample size saturation can lead to:

• Prioritizing the obvious and manifest content; missing latent, less obvious themes; and hampering the exploration of new topics
• Making superficial interpretations that ignore the context and variations in participants
• Projecting results to the population of interest based on a specific segment within that population

Although qualitative research is on the rise, boosted by new technologies and interest in the customer and user experience (UX), it often faces skepticism in the C-suite.

Many decision-makers feel uneasy about making important go/no-go decisions based on small samples, especially if patterns are unclear or there is no data to confirm that the results represent a larger population.

What to Do

How should qualitative researchers go about this?

My first time attending a QRCA conference was in 2020. I felt the awesome energy the large number of qualitative researchers brought to the room and was surprised about the willingness to share knowledge about business practices and methodological approaches.

However, at times, it felt a little insular. Nowhere did I hear talk about the need for validation of qualitative research insights using quantitative methods. For someone who does both qualitative and quantitative, it was a little disappointing.

For many of the issues we explore with qualitative research, there are corresponding quantitative techniques we can use to confirm or reject hypotheses that emerge. Here are a few examples:

• Product Concept Testing: To validate the appeal and viability of new products and services. For more on this, check How to Use Qualitative and Quantitative Research in Product Development. 
• Positioning Concept Testing: To select the most effective positioning statements for your target audience. For more on this, check How Product Positioning Affects Product Evaluations.
• MaxDiff: To pinpoint the magnitude of preferences or their importance to your target audience. This can be applied to identify important new product features, components of customer loyalty programs, customer satisfaction drivers, preferences for marketing collaterals, brand associations, and much more. For more on MaxDiff, check the article Making the Case for MaxDiff. 
• Conjoint Analysis: To validate potential product configurations of interest, mimic realistic purchase decisions in the context of competitive alternatives, and understand the willingness to pay and optimal price points. For more on this, read the article Use Menu-Based Conjoint Analysis to Optimize New Products. 
• Market Segmentation: To validate segmentation criteria suggested by qualitative personas research and determine if there are really viable segments worthy of different product and marketing development strategies. For more on this, check Market Segmentation Is Key to Success. 
• A/B Testing: To validate preferences for different concepts developed to increase customer conversions.

Recommendation

I have seen how a little bit of validation can elevate qualitative research in the eyes of decision-makers. It tends to increase confidence in the insights and provide the best support for making decisions with financial implications. Consequently, I encourage all qualitative researchers to consider including some quantitative research in their research design proposals.

If quantitative research is not part of your skillset, I strongly suggest you find a partner who complements your skills. Collaboration is the winning formula.

This article was published on QRCA’s Qual Power Blog on January 27, 2021.