by Michaela Mora

Posted on July 15, 2010

Qualitative research is going beyond in-person focus groups and experiencing a revolution for the better. Jim Bryson, president of 20/20 Research recently did a great presentation at the Fort Worth monthly luncheon organized by the DFW AMA about the latest online qualitative research techniques.

Thanks to the development of new online platforms, the qualitative research field have seen an explosion of new online qualitative research techniques that makes it possible to collect data in ways we couldn’t before.

Among the new online qualitative research techniques, we now have:

• Bulletin boards
• Immersion IDI’s
• Mobile qualitative
• Quantitative-Qualitative hybrids
• Real time chats
• Research blogs
• Research communities
• Social networking monitoring
• Video journals
• Webcam focus groups
• Web-Intercepts/chats

These new online qualitative research techniques have often made qualitative research better, faster and cheaper (not always). Studies using these online qualitative research techniques can be deployed pretty quickly and provide immediate access to transcripts or videos for review.

As for making qualitative research better, Bryson rightly points out to the advantages of most of these methods:

• Geography: it allow us to reach a wide-range of people across different regions without travel
• Candor: due to the privacy and confidentiality of online, many participants feel more comfortable to discuss sensitive topics
• Convenient access: both clients and participants can participate whenever is more convenient to them and their locations
• Longitudinal capabilities: studies can be extended over time as long as needed to understand the groups of interest

Below are some of the most often used online qualitative techniques, their advantages, disadvantages, and applications according to Bryson:

Mobile qualitative research, according to Bryson, will definitely be part of  market research’s near future. This can be easy and comfortable for the participants, although for now is limited to text only. This technique can be used for reaching to difficult groups, send reminders about “homework” given to study participants, and do research at the point of consumption.

Another approach getting traction is hybrid research, where quantitative and qualitative research are combined in one data collection opportunity. 20/20 Research recently launched a new service called Quallink where participants start in a survey and then are enrolled in a qualitative study. Hybrid research can also be done using SurveyGizmo, which has the capability to integrate online surveys with chat sessions from iModerate.

The main advantages of a hybrid approach are:

• Immediacy: there is no lag between quantitative and qualitative data collection
• Can be very cost effective since cost incurred in recruitment, travel and focus group facilities can be eliminated
• Ability to do a deep-dive on the story behind the numbers
• Ability to segment qualitative data together with quantitative results

There is no question that qualitative research has come a long way and that all these new techniques make this field exciting and promising, but before you get carried away by all the excitement, don’t forget to have clear research objectives and evaluate if these techniques are a good fit for what you want to accomplish.

by Michaela Mora

As published on July 6, 2010 in the July 2010 issue of the Quirk’s Marketing Research Review.

The advent of user-friendly online survey tools in recent years has created the illusion that anybody can write a survey questionnaire. After all, how hard can it be? It’s like asking questions in a conversation, many think. However, there are many methodological issues to consider when creating a questionnaire if you want to gather high-quality data in a survey. The following are 10 issues that arise in survey design.

• DATA COLLECTION METHOD

Some questions may elicit different answers if asked in an online survey, a telephone interview, a paper survey or a face-to-face interview. While words in phone surveys or in-person interviews are given more importance because of the conversational format, visual design elements have a bigger impact in how questions are read and interpreted in online surveys. Be aware of the types of questions that are a good fit for online surveys.

• RESPONDENT EFFORT

There are questions that put a heavier burden on the respondent’s working memory and comprehension or are likely to elicit higher non-response if asked in different data collection modes. Experience tells us that asking a ranking question with 10 items over the phone can overwhelm respondents. In online surveys, rating questions in matrix format with a large number of items increases fatigue and boredom and often leads respondents to adopt a “satisficing” behavior. Satisficing occurs when respondents select the same scale-point to rate all items without giving them too much thought. They go for the most effortless mental activity trying to satisfy the question requirement, rather than work on finding the optimal answers that best represent their opinion.

• QUESTION WORDING

Formulating a question with the right wording so it accurately reflects the issue of interest is one of the hardest parts in writing questionnaires. You may have seen political polls getting different answers depending on how a question is crafted. Data errors can creep into a survey if we use unfamiliar, complex or technically-inaccurate words; ask more than one question at a time; use incomplete sentences; use abstract or vague concepts; make the questions too wordy; or ask questions without a clear task.

Another issue related to question wording is the risk of introducing bias by leading the respondent in a particular direction. I recently received a mail survey sponsored by the Republican Party to represent the opinion of voters in my congressional district and one of the questions was:

“Do you think the record trillion-dollar federal deficit the Democrats are creating with their out-of-control spending is going to have disastrous consequences for our nation?”

Could this question be more biased? The use of adjectives such as “record,” “out-of-control” and “disastrous” makes it really clear what the expected answer is and what the intentions of the sponsor are.

• QUESTION SEQUENCE

Questions should follow a logical flow. Order inconsistencies can confuse respondents and bias the results. For instance if you are measuring brand awareness and ask respondents to recognize brands they are familiar with before asking which brands first come to mind, you are rendering the results from the latter question worthless since respondents can’t avoid thinking of brands they just saw in the first question. This seems basic, but it happens.

• QUESTION FORMAT

  Questions can be closed-ended or open-ended. Closed-ended questions provide answer choices, while open-ended questions ask respondents to answer in their own words. Each type of question serves different research objectives and has its own limitations. The key issues here are related to the level of detail and information richness we need; our previous knowledge about the topic; and whether to influence respondents’ answers. For example, for closed-ended questions we need to decide what the answer choices should be and in which order they should appear. This requires we know enough about the topic to provide answer options that capture the information accurately.

• INFORMATION ACCURACY

Some questions yield more accurate information than others. Respondents can answer questions about their gender and age accurately, but when it comes to attitudes and opinions on a particular issue, many may not have a clear answer. Overall, attitudes and opinion questions should be worded in a way that best reflects how respondents think and talk about a particular issue so that we can tease out information that is difficult for the respondent to articulate. However, some questions need to be skipped when they don’t apply to the respondents’ experience or the issue is so irrelevant to the respondent that s/he doesn’t have a formed opinion about it. In the case in which attitude statements appear grouped in a matrix format and some may not apply to a respondents (e.g., a customer satisfaction survey after a phone call to customer support), it is necessary to include a “Not sure/Don’t know/Not applicable” option to avoid introducing measurement error in the data.

For intance, the other day I received an online customer satisfaction survey from BlackBerry after a call I made to its support desk. The survey had a question in which I was asked to rate the representative who took my call on different aspects. One of them was “Timely Updates: Regular status updates were provided regarding your service request.” I wouldn’t know how to answer this, since the issue I called for didn’t require regular updates. Luckily, they had a “Not applicable” option, otherwise I would have been forced to lie, and one side of the scale would be as good as the other.

• MEASURED BEHAVIORS

People tend to have less-precise memories of mundane behaviors they engage in on regular basis, and usually they do not categorize events by periods of times (e.g., week, month and year). We need to consider appropriate reference periods for the type of behavior we want to measure. Asking “Have you purchased any piece of clothing in the last seven days?” will yield a more accurate behavior measure than asking “Have you purchased any piece of clothing in the last six months?”

Measured behavior should be relevant to the respondent and capture his or her potential state of mind. This is valid particularly when we use rating questions and have to decide whether to include a neutral mid-point. A lot of research has been conducted in this realm, particularly by psychologists concerned with scale development, but no definitive answer has been found and the debate continues. Some studies find support for excluding it while others for including it depending on the subject, audience and type of question.

Those against a neutral point argue that by including it we give respondents an easy way to avoid taking a position on a particular issue. There is also the argument that equates including a neutral point to wasting research dollars, since this information would not be of much value or at worst it would distort the results. This camp advocates for avoiding the use of a neutral point and forcing respondents to tell us on which side of the issue they are.

However, consumers make decisions all day long and many times find themselves idling in neutral. A neutral point can reflect any of these scenarios: we feel ambivalent about the issue and could go either way; we don’t have an opinion about the issue due to lack of knowledge or experience; we never developed an opinion about the issue because we find it irrelevant; we don’t want to give our real opinion if it is not considered socially desirable; or we don’t remember a particular experience related to the issue that is being rated.

By forcing respondents to take a stand when they don’t have a formed opinion about something, we introduce measurement error in the data since we are not capturing a plausible psychological scenario in which respondents may find themselves. This is yet another reason to include a “Not sure/Don’t know/Not applicable” option in addition to a neutral point.

• QUESTION STRUCTURE

Questions have different parts that must work in harmony to capture high-quality data. These are the question stem (e.g., What is your age?), additional instructions (e.g., Select one answer) and response options, if any (e.g., Under 18, 19 to 24, 25+). The wrong combination can leave respondents baffled about how to answer a question. Consider the examples below.


Overlapping answer options


What is your household income? Select one answer.


1. Under $25,000
2. $25,000 to $50,000
3. $50,000 to $75,000
4. $75,000 +

So, which answer should I choose if I my household income is $50,000? Is it option two or option three?


Conflict in meaning between different parts of the question


Please indicate the products you use most often. Select all that apply.


1. Cell phone
2. Toaster
3. Microwave oven
4. Vacuum cleaner
• VISUAL LAYOUT

Using design elements in an inconsistent way can increase the burden put on the respondent in trying to understand the meaning of what is asked. For example, encountering different font sizes and colors across questions forces the respondent to relearn their meaning every time they are used.

Also, presenting scales with different directions (positive to negative or vice versa) in rating questions within the same survey increases measurement error as respondents often assume all rating questions have the same scale direction even when the instructions explain the meaning of the end points of the scale. For instance, if a preference question using a 1-7 scale where 1 means “the most preferred” is followed by an importance question, also using a 1-7 scale, but where 1 means “the least important,” respondents who are not paying attention to the instructions (which is quite common) are likely to assume that the 1 in the importance question means “the most important.” I have seen many examples of this problem, when respondents are asked a follow-up question conditioned on their previous answers and then they realize their mistake and tell us they actually meant to say the opposite.

• ANALYTICAL PLAN

Based on the research object, both the type of information requested and the question format are important for the type of analysis we plan to perform once the data is collected. If you want to develop a customer satisfaction model using linear regression analysis and the dependent variable is an open-ended question, you can forget about modeling anything. This seems obvious, but I have seen non-researchers writing questionnaires without thinking how they will analyze the data and then come to me asking for analyses that are not appropriate for the data collected.

There is also the question of whether we want to replicate the results, track certain events or just run a one-time ad hoc analysis. If the goal is to track certain metrics, time and care should be dedicated to crafting tracking questions, as slight changes in wording can change the meaning of a question and thus its results.

ON YOUR WAY

If you take each of these aspects of survey writing into consideration, you will be on your way to creating surveys that produce valid data and can support with confidence strategic and tactical decisions for your business.

by Michaela Mora

Posted on June 9, 2010

I recently got an inquiry from a SurveyGizmo user asking about what response rate he could expect from using this online survey tool. Fortunately for any online survey tool, including SurveyGizmo, response rates to online surveys don’t depend on the survey tool you use.

First let’s distinguish between response rates, incidence rates, completion rates and non-response. They are related, but not the same, and some clients use these concepts interchangeably, which lead to confusion in sample size and cost estimations.

Response rates  are usually calculated based on the number of respondents who attempt to participate in a survey, even if they are disqualified after they have been screened with certain questions. If we send a survey invitation to a sample size of 100 people and only 5 attempt to take the survey, then the response rate would be 5%.  Response rates have been used for years as indicators of data accuracy, however recent research has indicated that lower response rates don’t necessarily mean low quality data.

Response rates are affected by:

• Survey topic relevancy: People will not dedicate time to participate in surveys that are perceived as irrelevant.

• Incentives: Sometimes an incentive is needed to motivate respondents, but careful consideration needs to be given to this. Incentives are a tricky subject since we may attract only certain types of respondents and insert selection bias in the sample.

• Survey invitation: Survey invitations should be personalized and provide compelling reasons to participate in the survey. A poorly written invitation can drive respondents away or not catch their attention. Use appealing subject taglines and make the invitation short, clear and persuasive.

• Type of relationship with target survey audience: Depending on the level of relationship respondents have with the brand, organization or company sponsoring the project they will be more or less motivated to participate. For example, customer surveys tend to have higher response rates than those targeted at non-customers. For more on this, check Survey Response Rate Directly Proportional to Strength of Relationship by Jeffrey Henning.

• Privacy protection concerns: People are not comfortable sharing information if they don’t know how it is going to be used. Communication about privacy policy and data security should be clear.

• Reminders: These may be needed to reach busy people or those not available  within a certain time frame when the first invitation is sent out.

Incidence rates are based on the number of respondents that qualify for a study based on certain screening criteria. For example, if we need a sample of females in the general population without any other requirements, the incidence rate is expected to be 50% since half of the population are women. Incidence rates will vary depending on who we are targeting with the study.

Response rates are often used to indicate the number of completed surveys, but I think it is worth to make the distinction between response rates and completion rates since this has methodological and cost implications ( e.g.  when we need to purchase sample from online panel providers).

Completion rates indicate how many people who qualified for the study completed the survey. If they enter the survey, answer some questions and then abandon the survey, they will be counted as incompletes and are usually excluded from the final data. The number of incompletes increases when:

1. The survey is too long
2. Survey flow is confusing
3. There are skip logic errors that show irrelevant questions to respondents who can’t answer them
4. Questions are poorly worded and instructions are unclear
5. Questions are complex and requite a lot of mental effort from the respondent
6. The respondent is not rewarded accordingly based on survey length and amount of effort required
7. The topic and survey format can’t hold the respondent’s interest
8. Privacy protection is unclear or lacking

Non-response occurs when we fail to get a response from the total sample either because respondents refuse to participate in the survey or they start but never complete it. If non-responses follow a pattern that systematically excludes a particular segment of the sample, they introduce what it calls selection bias, which will prevent us from getting a representative sample of opinions in the population of interest. Nonrespondents are often different from respondents, so their absence in the final sample can make it difficult to generalize the results to the overall target population.

In short, regardless of the survey tool you use, you can improve response rates and completion rates if you avoid most of the problems mentioned above.

OTHER RELATED ARTICLES

• Does A Large Sample Size Guarantee A Representative Sample?

• What Is The Right Sample Size For A Survey?

• Testing For Significant Differences In Convenience Samples – What Is The Point?
  

by Michaela Mora

Posted on May 20, 2010

I meet many clients who worry about sample size  trying to ensure they get an enough large sample so that statistically significant differences can be found and inferences to a larger population can be made, but they often don’t know that these statistical tests were meant to work within the probability sampling theory framework.

Since the advent of online panels and the increase of online surveys using panel-provided samples, the issue of testing for significant differences using standard parametric tests has become a moot point in many research studies.

Nowadays many of the surveys conducted online use samples provided by online panels, but these are mostly convenience samples (non-probability). The populations of online panels include respondents who are willing to participate in studies, excluding those unwilling to be part of the panel who may be members of the target population we are after.

In probability sampling, each possible respondent from the target population has a known probability to be chosen. Probability sampling helps us to avoid some of the selection biases that can make a sample not representative of the target population. For more on this read Does A Large Sample Size Guarantee A Representative Sample?

A single probability sample doesn’t guarantee to be representative of a target population, but we can quantify how often samples will meet some criterion of representativeness. This is the notion behind confidence intervals. The probability sampling procedure guarantees that each unit in the population of interest could appear in the sample.

By taking into account all possible random samples that can be taken from a population, we can estimate how often the true value of an estimate can be expected to be within a specific range of values. So, when we  talk about a 95% confidence interval, this really means that the true value of a particular variable is expected to fall within an interval of values 95  out of 100 times we repeat the procedure. When an opinion poll indicates that 50% of people are in favor of a political decision with a +/-3% margin of error at a 95% confidence interval, it is really saying that we can expect that between 47% and 53% of people will be in favor of the decision 95 out 100 times, if we were to repeat the poll. When we test for significant differences, we are looking to see if the value falls outside that range.

Unfortunately, taking a probability sample is hard and costly. For most consumer research studies and social behavior studies, we really don’t know the size of the actual population of consumers behaving in certain ways or consuming certain products, and trying to find out would make the research prohibitively expensive. This is why we often have to settle for convenience samples like the ones offered by online panels. They still can offer valuable insights if designed with care, but again doing statistical testing in a convenience sample is pointless since the assumptions about probability sampling are violated.

Online panels are here to stay, and they will continue to be a source for affordable sample for market research. Research using convenience sample is often better than not research at all if the survey is well designed and screening criteria are used to define the target population.

A more appropriate case for testing statistically significant differences are random samples taken from a customer database, since this is essentially the population frame where we can count all members and estimate their probability to be chosen.

 However, if you don’t have a customer database or are interested in surveying non-customers, then  use a convenience sample, if that is what your research budget can afford or there is no other way to get to the actual population frame (list to pull the sample from), but don’t fret about testing for significant differences. You may feel more confidence if you are able to replicate the results in repeated surveys, but be always cautious about inferences made from convenience samples since there could be a hidden systematic bias in the data.

It is always important that whenever you use convenience samples  you consider the following when analyzing the results:

         1. Who is systematically excluded from the sample?

         2. What groups are over- or underrepresented in the sample?

         3. Have the results been replicated with different samples and data collection methods?

If testing for significant difference gives you peace of mind, even when using convenience samples, do it to confirm the “direction” of the data, but restrain yourself from doing inferences to a larger population.

OTHER RELATED ARTICLES

• Does A Large Sample Size Guarantee A Representative Sample?

• What Is The Right Sample Size For A Survey? 

by Michaela Mora

Posted on May 13, 2010

I often get asked “What sample size do I need to get a representative sample?” The problem is that this question is not formulated correctly. 

Sample size and representativeness are two related, but different issues. The sheer size of a sample is not a guarantee of its ability to accurately represent a target population. Large unrepresentative samples can perform as badly as small unrepresentative samples.

A survey sample’s ability to represent a population has to do with the sampling frame; that is the list from which the sample is selected. When some parts of the target population are not included in the sampled population, we are faced with selection bias, which prevent us from claiming that the sample is representative of the target population. Selection bias can occur in different ways:

• Convenience sample: This includes respondents who are easier to select or who are most likely to respond. This sample will not be representative of harder-to-select individuals. Samples from online panels are a good example of convenience samples. These panels are composed by individuals who have expressed interest in participating in surveys, leaving out individuals who may be part of the target population, but are not available for interviewing through the panel.

• Undercoverage: This happens when we fail to include all the target population in the sampling frame. Many online panels work hard at avoiding undercoverage bias, but the fact remains that certain demographics are underrepresented. For example, it is difficult to field online studies targeted at the total Hispanic population in the US without using a hybrid data collection approach that allows us to reach unacculturated Hispanics, who are usually underrepresented in most online panels. Coverage bias is also found in phone surveys that use telephone list sampling frames that exclude households without landline access. As more households substitute cell phones for their landlines, obtaining representative samples of certain demographic groups will soon be difficult without including cell phone lists in the sampling frame.

• Nonresponse: Selection bias also takes place when we fail to obtain responses from all respondents in the selected sample. Nonrespondents tend to differ from respondents, so their absence in the final sample makes it difficult to generalize the results to the overall target population. This is why the design of a survey is far more important than the absolute sample size to get a representative sample of the target population.

• Judgment sample: This is a sample selected based on “representative” criteria based on prior knowledge of the topic or target population. An example would be a study looking for a sample of teenagers, and trying to intercept them at a cross-section near a high school.

• Misspecification of target population: This happens when we use intentionally or unintentionally screening criteria that leave out important subgroups of the population.

• Poor data collection quality: This can introduce selection bias when there are poor quality controls to ensure that we interview the designated members of the sample. An example of this include allowing whoever is available in the household to take the survey instead of the intended member based on certain screening criteria.

So when it comes to getting a representative sample, sample source is more important than sample size. If you want a representative sample of a particular population, you need to ensure that:

1. The sample source includes the whole target population
2. The selected data collection method (online, phone, paper, in person) can reach individuals, with characteristics typical of those possessed by the population of interest
3. The screening criteria truly reflect the target population
4. You can minimize nonresponse bias with good survey design, incentives and the appropriate contact method
5. There are quality controls in place during the data collection process to guarantee that designated members of the sample are reached.

The initial question about sample size versus representative sample is usually followed by “What sample size do I need to get statistical significance? For an answer to this question, check my previous articles What Is The Right Sample Size For A Survey? and Testing For Significant Differences In Convenience Samples – What Is The Point?

For help on sample size calculation use our Sample Size and Margin of Error Calculators.

by Michaela Mora

Posted on May 6, 2010

Determining the sample size is one of the early steps that must be taken in the planning of a survey. Unfortunately, there is no magic formula that will tell us what the perfect sample is since there are several factors we need to think about:

• ANALYTICAL PLAN: The research objectives and planned analytical approach should be the first factor to consider when making the decision on sample size. For instance, there are statistical procedures (e.g. regression analysis) that require a certain number of observations per variable. Moreover, if comparative analysis between subgroups in the sample is expected, the sample size should be adjusted for it to be able to identify statistically significant differences between the groups.

• POPULATION VARIABILITY: This refers to the target population’s diversity. If the target population exhibits large variability in the behaviors and attitudes of interest being researched, a large sample is needed. If 20% or 80% of the population behaves in certain way, this indicates less variability than if 50% would do so. To be conservative, it is standard practice to use 50% (0.5) as the event probability in sample size calculations since it represents the highest variability that can be expected in the population.

• LEVEL OF CONFIDENCE: This is the level of risk we are willing to tolerate usually expressed as a percentage (e.g. 95% confidence level). Although survey results are reported as point estimates (e.g. 75% of respondents like this product), the fact is that since we are working with a sample of the target population, we can only be confident that the true value of the estimate in that population falls within a particular range or what is called confidence interval. The level of confidence indicates the probability that the true value of the estimate in fact will fall within the boundaries of the confidence interval. How confident can you be? As confident as your tolerance for risk allows you to, knowing that the confidence level is inversely proportional to estimate accuracy or margin of error. The more confident you want to be, the larger the confidence interval that is needed, which leads to lower levels of precision.

• MARGIN OF ERROR: Also known as sampling error, indicates the desired level of precision of the estimate. You have probably seen poll results quoted in the media, saying that the margin of error was plus or minus a particular percentage (e.g. +/-3%). This percentage defines the lower and upper bounds of the confidence interval likely to include the parameter estimate, and it is a measure of its reliability. The larger the sample, the smaller the margin of error and the greater the estimate precision.

Below is a table illustrating how the margin of error and level of confidence interact with sample size. To get the same level of precision (e.g. +/-3.2%), larger samples are needed as the confidence level increases. For example, if we want to be certain that in 95 out of 100 times the survey is repeated the estimate will be +/- 3.2%, we need a sample of 950.

OTHER RELATED ARTICLES

• Testing For Significant Differences In Convenience Samples – What Is The Point?

by Michaela Mora

as published on April 30, 2010 by the Dallas Business Journal

Bad economic news has been a daily event for the last 24 months. We have heard about thousands of companies filing for bankruptcy and millions of people losing their jobs. Last year, most companies slashed marketing expenses and reduced market research budgets to almost nothing. It’s survival time, but I would argue many companies are going about it all wrong. Now, more than ever, market research can help companies to invest wisely the few resources they have. Ideally, this is the time to get creative and find cost-effective ways to conduct market research.

So, how can you do good market research on a small budget? Here are points to consider:

• CONTROL FOR RESEARCH SCOPE. Look long and hard at the business issues at hand and separate “nice-to-know” from “need-to-know” information. Focus on the most critical issues to reduce survey length and time needed for data processing and data analysis.

• REDUCE SAMPLE SIZE. ample size requirements should be carefully considered, taking into account the impact on estimate accuracy. As samples get larger, estimate accuracy and sample costs increase, but there is a point where the gains in estimate accuracy don’t justify the increased cost. Recently, I had a client who wanted to conduct an online survey with a sample of 1,000 respondents, which would give a statistical accuracy of +/-3.1%, but it would cost $8,000. On the other hand, a sample of 400 respondents would give a statistical accuracy of +/-4.9% and cost $3,400. As you can see, a 135% increase in sample cost would only yield a 60% gain in statistical accuracy. The client opted for the smaller sample.

• USE CUSTOMER DATABASES. Customers’ feedback often provides invaluable insights given the relationship they already have with a company. What’s more, a customer database is the cheapest source of sample for research purposes, since it is already in-house. However, be aware of its potential limitations given the profile of your customers and the type of data you have gathered about them. Know how the database was sourced. For example, if most of your customers are small businesses or females younger than age 25 and you are interested in understanding how your products will perform in other market segments, then your customer database is not the right sample source for a market segmentation study.

• USE ONLINE DATA COLLECTION TOOLS. In the past few years an explosion of online survey tools has reduced the cost of data collection significantly and shortened the research process time line. However, there are a couple of caveats:

 Market research is the field in which the principle of “garbage in, garbage out” has the most application. Programming and deploying online surveys is easier and cheaper than ever, however the online tools that facilitate this don’t provide the content of the actual survey. You still need expertise in questionnaire design, research methodology, sampling and data analysis to be able to extract useful insights from the survey data.

Online surveys are not always the most appropriate methodology for the research objectives and target sample. For example, consumers in lower income brackets are less likely to have Internet access, therefore they would be underrepresented in an online-only sample. In this case, it would be more appropriate to reach them by phone or in person.

• HIRE SMALL MARKET RESEARCH AGENCIES. There are many small market research agencies with big agency capabilities and experienced market researchers, but low overhead cost. They are often very responsive and flexible and can do the job with the same — and sometimes better — quality than bigger research firms for a fraction of what larger firms would charge.

Market research doesn’t have to be expensive to be well done. Consider these points and find a research partner that is willing to invest in your success. This will allow you to do market research on the smallest budgets.

by Michaela Mora

Posted on March 12, 2010

In my previous article about how to connect to your customers with the help of research insights, I listed some of the key research questions that any business wanting to grow and succeed should ask. But, how do you do it?

Here we need to make a distinction between data collection methods and types of research based on analytical approach, which are often confused. Data collection methods differ based on whether we want to conduct quantitative or qualitative research.

Qualitative research, which is exploratory in nature, usually uses data collection methods such as focus groups, triads, dyads,  in-depth interviews, uninterrupted observation, bulletin boards, and ethnographic participatory observation.

Quantitative research, which looks to quantify a problem, collects data through surveys in different modalities (online, phone, paper), audits, points of purchase (purchase transactions), and click-streams.

As for types of research, I mean the approaches used to analyze the data collected. Depending of the business objectives, we may decide to gather data to conduct a market segmentation, product testing, advertising testing, key driver analysis for satisfaction and loyalty, usability testing, awareness and usage research, and pricing research, among others.

 When to use each of these data collection methods and types of research depends on the business issues we are dealing with in one or more of four key areas:

• Awareness: Let the market know that the product or service exists
• Targeting: Reach the target segments with the highest profit potential
• Acquisition: Optimize the marketing message, offer,  and price that will close the sale
• Retention: Generate repeat purchases from current customers

The chart below, which we call the Relevant Wheel, shows when it is most appropriate and relevant to conduct different types of research.

Our clients find this chart to be helpful and use it as a reference to determine when a particular type of research is needed. Once this is defined, we discuss the most appropriate qualitative or quantitative data collection methods.

 So next time you wonder what type of research to conduct, I invite you to ask yourself where the particular problem at hand belongs to (Awareness, Targeting, Acquisition or Retention ) and then use this chart to to guide your decision on the most appropriate type of research. I hope you find it useful.

by Michaela Mora

Posted on March 10, 2010

I often meet entrepreneurs who have started, or are starting, a new business and who ask me what type of market research they should conduct. Some are inventors who have come up with a gadget they love, but don’t know if there is market for it. Others have launched products and services and soon realize they need to re-think product features, benefits, advertising and the customer experience.

Regardless of whether your customers are consumers or business decision makers, at Relevant Insights we always recommend assuming a customer-centric market research approach in alignment with our clients’ business objectives. This is based on two basic principles:

• Learn how to connect with your target customer by getting a 360 degree view of his behaviors, experiences, knowledge and emotions.




• Have a clear idea of what you want the customer to think and do regarding the company’s brand and products having in mind the ultimate business goals: acquire and retain customers.

A market research plan based on these two principles would include four key areas of research, which become relevant depending on the internal knowledge available inside the company in connection with the lifecycle of the product. These four areas are:

• Awareness: As a Cuban saying goes “the baby who doesn’t cry, doesn’t eat,” we have to let the market know about our products and services if we want buyers.




• Targeting: Businesses usually are more successful if they are able to define early on who their target customers are, and which customer segments have the highest profit potential. 




• Acquisition: Without customers any business is bound to fail, so we need to know how we can attract new customers through appealing product benefits, the right pricing model, and effective marketing programs.




• Retention: Generating repeat purchases from current customers is the most cost-efficient path to business growth, so a retention strategy is a must for any business that wants to thrive.

There are many research questions that can be asked within each of these area depending on the stage a product, brand or company is at. Below are some of the questions that should be in the short list of any business that wants to succeed.  

Find the answers to these questions and you will learn how to connect with your target customers and be light years ahead of your competition.

by Michaela Mora

Posted on February 9, 2010

I recently came across the new ad from Domino’s Pizza where they show a clip of focus groups they conducted with consumers about their products. I love it! The message was clear: they listened to their customers. Their management and product teams were brave enough to really pay attention to what customers think. I’ll be eternally grateful to Domino’s pizza for the message sent about the value of market research.

This may not sound as a novel idea, but many, many companies go about their business thinking they don’t need to conduct market research in order to improve their products and grow. They believe they know enough about their industry and product category that there is nothing new to learn. Then there are companies that are barely aware of the importance of research, but see it as an expense and not an investment. They prefer to throw spaghetti at the walls and see what sticks.

I don’t know how many focus groups Domino’s did or if they also used other research methods to test their improved pizzas, but the important thing here is that they were willing to listen to their customers.

Now regarding methodology for new product development, I always advise clients to combine qualitative (e.g. focus groups) and quantitative research (e.g. surveys) methods.

Qualitative research is by definition exploratory, and it is used when we don’t know what to expect, to define the problem or develop an approach to the problem. It’s also used to go deeper into issues of interest and explore nuances related to the problem at hand.

Quantitative research is conclusive in its purpose as it tries to quantify the problem and understand how prevalent it is by looking for projectable results to a larger population.

Here are some guidelines to use both types of research in new product development:

Combining both approaches when developing new products will give you a solid foundation to make the right decisions for your business grounded in consumer insights.