How To Do a Research Problem Audit

Problem definition is the most important step in the market research process. A problem audit can help define it. If a problem is misunderstood or ill-defined, all money, time, and effort spent on the research will be wasted. Serious mistakes are made when we try to act on answers to the wrong research questions.

In many cases, decision-makers have a vague idea of the problem when they ask for research. They often focus on the symptoms of a problem (e.g., high customer churn, low market share, unmet sales forecasts, etc.), but don’t know the underlying causes.

Only when underlying causes are identified can problems be successfully addressed.

For example, loss of market share can be caused by promotions from competitors, inadequate distribution, bad customer experiences due to an inadequate service blueprint, negative brand reputation due to company actions, technology changes bringing new competition, lack of a diverse product portfolio, etc.

It is not uncommon that the research takes a different direction after we take steps to clarify and define the problem in order to select a fit-for-purpose approach.

Problem Definition Tasks

To define the research problem, we often need to take on a series of tasks to help researchers understand the background of the business problem and the environmental context.

The main goals of these tasks are:

1. Identify what problem the decision-maker is facing – What course of action should be taken?
2. Translate the decision-maker problem into a research problem – What information is needed to justify the selected course of action?

The most common tasks involved in problem definition are:

• Discussions with decision-makers and various stakeholders connected to the problem area
• Review of secondary research
• Interviews with subject matter experts
• Qualitative research with customers and non-customers (e.g., In-depth interviews, focus groups, etc.)

Discussions with Decision Makers

Working with decision-makers on defining the problem is crucial to the success and value of the research. Decision-makers need to understand the capabilities and limitations of the research. Research provides information relevant to decisions, but decision-makers are ultimately responsible for the solutions.

By the same token, researchers need to understand the nature and impact of the decisions stakeholders responsible for solutions face.

Interacting with decision-makers may be difficult in some organizations. Lack of time, long chains of command, and internal politics may make it difficult for researchers to talk directly to decision-makers with the must say on how to implement the insights of the research.

Conflicting interests in the research outcomes from various internal groups may complicate the matter further, so communication skills and some planning are often required to engage all the stakeholders connected to the problem.

A key component of the planning is to create a discussion guide to conduct a problem audit.

Problem Audit

A problem audit is an evaluation framework that helps us interact with decision-makers to uncover what the real problem is and the areas of research we need to focus on to identify its underlying causes. The main purpose is to understand the issues the decision-makers are facing.

The actionable problem audit framework includes questions in seven major areas.

History of the problem

What events led to the realization that action is needed? Understanding what happened or has been happening that is forcing stakeholders to look for new solutions is the first step in a problem audit in market research or UX Research.

For example, the company may be losing market share to new competitors over time, despite various marketing initiatives. A key competitor may have launched a new product with an aggressive marketing campaign with promotional offers and made the problem acute.

Available Alternative Courses of Action

What solutions to the problem are currently available? Decision-makers may have considered courses of action that are available or could be available shortly. However, these courses of action may pose risks, be riddled with limitations, or be in early stages that need research to validate their viability.

Following the example above, the company could introduce new products still under development, create quickly me-too products, reduce prices, offer different promotions, launch a new campaign to reposition the brand, etc.

Criteria to Evaluate Alternative Courses of Action

What criteria and metrics are decision-makers considering making go/no-go decisions to select viable solutions?

Criteria may vary from problem to problem depending on tolerance for risk and the impact of a solution (financial, operations, personal, etc.).

For each of the currently available solutions, the company in our example could use criteria such as thresholds for sales, customer experience ratings, customer retention rates, customer acquisition rates, cost savings, profitability, brand awareness, etc.

Potential Actions that Research Findings May Suggest

What could be potential research findings, if we do the research, and what course of action would those suggest? Would they be tactical or strategic decisions?

While the previous audit areas are about what we know so far, the questions in this one deal with hypotheticals to expand our knowledge area.

Here we can discuss hypotheses that need validation. These may call for preliminary research using secondary data sources or discussions with industry and subject matter experts.

In our imaginary company, we could explore the idea that new market opportunities may exist in adjacent product and service categories not offered by competitors.

Information Needed to Answer Decision Makers’ Questions

What information do we need to answer the different questions related to the problem and its underlying causes?

As the problem is discussed, its history, potential available solutions, risks, limitations, etc. the problem itself may morph into a different problem or the focus may go to related areas where the real causes may lay. This part of the discussion guide is the most fluid and requires keeping in mind all the issues discussed so far. We may need to define the information needs through various iterations of these discussions.

In my experience, when we make a list of the information needs and decision-makers see it in writing, corrections and adjustments are made in increments when they see it articulated in concrete statements after the discussions.

This is an excellent opportunity for consensus-building exercises, so stakeholders and researchers become aligned on the research objectives.

Continuing with our example, we may have decided that the problem is not that we are losing customers, but that we are not acquiring new ones and a new generation of customers may be more interested in adjacent services (hopefully this conclusion comes from secondary research and or qualitative research).

In this case, the information needs for the current research being discussed could be related to understanding who the competitors in those adjacent product categories are, how these products and services are purchased and used, what the perceived benefits are, and how this offering will impact our brand positioning and pricing strategy.

Decision Makers’ Use of Information Vs. Intuition

How will decision-makers use the information from the research to make decisions? Do they tend to rely on evidence-based insights or their own intuition and judgment?

A lot of business decisions are based on “gut feeling,” especially if the research results go against expected conclusions. Confirmation bias wins the fight most of the time unless decision-makers are open to new perspectives.

Researchers need to consider the strength of confirmation biases among stakeholders involved in the research and call attention to how they may influence the problem definition, the information needs, and the research approach selected. Confirmation bias is likely to limit the search for new solutions to the problem.

Decision-Making Corporate Culture

In some organizations, decision-making is driven by processes, while in others is dominated by the personalities of key decision-makers.

Decision-making corporate culture is an important factor in the perceived value of the research and requires that researchers sometimes get into the thick of internal politics to be able to design actionable research.

In organizations with a decentralized and more democratic style of decision-making, researchers have to deal with many stakeholders and often play a facilitator role to bring stakeholders together, if there are no processes in place delineating decision-making steps (e.g., committees, requests forms, etc.).

In organizations where key decision-makers set the tone, regardless of processes, researchers need to find ways to involve them in the problem definition and solution development and understand how to best present research results to this audience so the insights are not disregarded in the battle with “gut-feeling” and anecdotal evidence.

In Short

Market researchers and UX researchers should be prepared to get research requests for which the problem is not well defined at best or misunderstood at worse. Do not trust that internal stakeholders have a clear idea of what the problem is. Run problem audit and probe!

It is our role to help define the business problem (focus on the action) clearly to facilitate the translation into the research problem (focus on information needs) and the selection of a fit-for-purpose research design and methodologies.