The 2015 MSMR Alumni Research Conference that took place recently at the University of Texas in Arlington (UTA), opened with a presentation by Susan Schwartz McDonald from NAXION, in which Dr. Schwartz tried to summarize her 35 years of experience doing market research in ten key learnings. I found myself nodding at each point because she described what many professional market researchers, including me, have experienced.
I kept think that these nuggets of wisdom needed to be shared with those coming to the market research field either as providers or users, so here they are:
1. KNOW THE DIFFERENCE BETWEEN BUSINESS PROBLEMS AND RESEARCH PROBLEMS, AND FRAME THEM WELL. Business problems are not the same as research problems. Research questions exist in the context of business problems and should provide answers to solve them. The question Which product is preferred? is a research one. Its answer can be used to solve business problems such as In which product should we invest? Where are the growth opportunity for our business, How do we beat our competition?, among others. Both providers and users of market research need to understand the business problem to frame the research questions correctly and provide relevant answers.
2. DO NOT WASTE RESOURCES IN RESEARCHING WHAT IT’S ALREADY KNOWN. Fear and internal politics have been behind a lot of market research done to validate what’s already known. There is knowledge in the organization that sometimes is discarded and new research is requested as part of a cover-your-ass strategy. Believe me, I’m all for using research for hypothesis validation, but for some questions research is simply not needed.
3. WORK HARD TO BECOME A PROFESSIONAL MARKET RESEARCHER. The easy access to online survey tools have given the idea to many outside the field that no special skills and knowledge is needed to work as a market researcher. However, as Dr. Schwartz said “Amateurism has contributed to cynicism about survey research.” There are a lot of bad surveys out there producing garbage data just because those designing them don’t have the most basic knowledge about the principles of market research. If you are new to the field, please find training or hire professional market research to help you.
4. FOCUS ON THE INPUT MORE THAN ON THE OUTPUT WHEN IT COMES TO DEVELOPING TOOLS. Our field is extremely sensitive to the garbage-in-garbage-out principle. Unfortunately, there is a race to develop tools that can capture, process, and present data often based on concerns more about the output than the input. For example, I have met many clients to whom online survey tools have given the impression that user-friendly tools equate to good surveys. The tools may have many features to develop good surveys, but the tools don’t write the surveys themselves. Market researchers are responsible for that, and designing good surveys seems easier than it is. Badly designed surveys, as Dr. Schwartz says, make your tools look bad.
5. LOOK TO MINIMIZE MEASUREMENT BIASES. All market research methods are susceptible to what is called the Observer Effect in which the measurement of a phenomenon is affected by the measurement itself. A simple example of this can be found in the order in which we ask questions, as each question can potentially prime the response to the next, and the results can be very different when the order is changed. This is why we have to be cognizant of the advantages and disadvantages of different data collection and analysis methods to minimize measurement errors.
6. ASK QUESTIONS PEOPLE CAN ANSWER. It has become a trend to despise survey research under the belief that people can’t articulate what they want, can’t remember behaviors, etc. The things is that people can provide useful information only if you ask questions they can really answer, and many surveys have questions that don’t allow for that. Some clients, due to budget constraints, short timelines, or simply lack of knowledge want to use surveys for everything, but this method is not a good fit for all types of business and research questions. Again, knowledge of market research principles and methods is needed to select the best approach and design the right types of questions in order to provide useful and relevant information
7. BE AWARE OF THE DIFFERENCE BETWEEN STATISTICAL SIGNIFICANCE AND PRACTICAL SIGNIFICANCE. Focusing on statistical significance can be misleading if we don’t take into account the practical significance of the results. Take market segmentation for example. A solution can uncover significantly different, but very small segments that would be enormously costly to reach. Sometimes, clients only want to do research in certain niche segments without understanding them in the context of the general population, missing invaluable insights into the practical significance of targeting such segments. If you find statistically significant differences, it doesn’t always mean they have practical value.
8. DON’T FALL IN LOVE WITH ONE SINGLE MARKET RESEARCH TECHNIQUE. This is probably the most dangerous kind of love in the market research field, often driven by level of comfort and knowledge of the many different tools available to market researchers. We need to be open to try new techniques as the business and research problems dictate, without ignoring others tried and proven. Each research approach, qualitative or quantitative, provides different types of insights. As Dr. Schwartz said, “research fashions come and go, but what you need is a whole wardrobe.”
9. DON’T DISCOUNT WHAT YOUR CUSTOMERS SAY WHEN STATISTICAL MODELS DON’T AGREE WITH THEM. We know that adding more variables increases model fit, so use hold-out samples for validation and be aware of over-fit models. I agree when Dr. Schwartz says that customer insights are only as trustworthy as your data and your willingness to hear bad news. When things don’t match, check your math and your intellectual honesty
10. USE MARKET RESEARCH TO LEARN HOW TO THINK LIKE YOUR CUSTOMERS. Users of market research are not exempt from confirmation bias; that’s, paying attention mostly to what they want to see or hear. As Dr. Schwartz advises “Don’t look only for answers, but seek to develop empathy.” We need to be aware of this bias and recognize when our interpretation of the results are driven by pre-conceived ideas, and it is robing us from really understanding our customers. We need to use market research to walk in our customers’ shoes, to really understand where they are coming from, to be able to provide products and services that are relevant to them.
In the end, Dr. Schwartz says, our mission as market researchers is to interpret, integrate, and extrapolate. We need to interpret the results in the context of the business problem and help clients make decisions. By the same token, clients need to be open, and consider market researchers as partners by collaborating and providing business insights that only someone inside an organization may have, otherwise interpretations are made in a vacuum, integration would not be possible, and extrapolations can be simply wrong.