Cause-effect research requires special research designs. However, many assume that surveys can uncover cause and effect links without much consideration.
Not long ago I got a call from a potential client asking for research to determine why a recent marketing campaign failed to increase sales, despite a significant increase in awareness.
He had conducted an advertising awareness survey, and the results showed that many in the target audience had noted the advertising and gave it high ratings, but didn’t make a purchase.
All possible explanations were mere speculations. He couldn’t pinpoint any particular cause for this.
The main problem was that he looked for evidence of a cause-effect link, but the research design was not appropriate for that.
The main method for cause-effect research is experimentation. In experimental-based research, we manipulate the causal or independent variables in a relatively controlled environment. This means that we control and monitor other variables affecting the dependent variable (e.g. sales) as much as possible.
In this case, the client had conducted the survey and analyzed the data without taking into account the effectiveness of different marketing collaterals, market penetration, competitor activity, and some characteristics of the purchase decision-makers.
After doing some digging around, we uncovered that in some markets, competitors had launched high-frequency advertising campaigns. This helped the client indirectly by increasing category awareness, but not his sales.
Moreover, the program targeted recent buyers who probably didn’t have a need for his products at that particular moment.
Surveys that are not part of an experimental approach may show correlations, but not causality.
To really connect the dots between cause and effect, we needed to create an experiment. This would include different renditions of the marketing collaterals, different markets, customers at different stages in the purchase cycle, and actions taken by competitors.
Experimentation in marketing has traditionally taken the form of standard test markets. In this approach, you launch controlled advertising in designated markets and sell the product through regular distribution channels.
However, these tests can be time-consuming, are often expensive, and may be difficult to administer.
Simulated test markets are a more affordable solution. In this approach, we expose individuals to the product or concept (e.g. via actual marketing collaterals), and give them the opportunity to buy it. If they buy it, we ask them to evaluate the product and state their repeat purchase intent.
We can then combine trial and repeat estimates with data about promotions, distribution levels, competitor activity, and other relevant pieces of information.
User Research (UX)
Another experimentation channel is the popular freemium model, which mimics this process to some extent. The basic principle is to let people try it and observe what decision they make.
After this, we can follow up with research to understand what drove their decision while controlling for other variables that may affect the outcome. This approach goes deeper into user research.
Experiments are the Answer
In short, if you want to understand cause and effect, you need to conduct experiments.
Experiments may include surveys as a data collection method, but surveys in themselves can’t provide the answer. It is the experimental design that will lead you to it.
(An earlier version of this article was originally published on January 18, 2012. The article was last updated and revised on July 29, 2019.)