Avoid These Survey Design Mistakes

 |  Posted: by

Avoid These Survey Design Mistakes

 

Start with a good survey design, if you want good survey data. I recently took a brand awareness and positioning survey in the yogurt category that left me wondering how useful my answers were to the brands in questions.

 

FIRST MISTAKE

 

After some screening questions, I got one about familiarity with different brands of yogurt, point at which I started questioning the survey quality. This question used  the following 3-point scale:

 

1. Never heard of

2. Heard of, but never used

3. Familiar

 

Although this scale is not consistent in the terms it uses and doesn’t flow with the terms used in the question, it is still easy to understand what it is after. The issue I had is that I didn’t know what being “familiar” with each of the brands really meant. Based on the context of the other two options, and without any others to indicate levels of familiarity, I assumed it equated “familiar” with “have consumed the brand before.”

 

SECOND MISTAKE

 

The problem with the survey design became pretty clear in the questions that followed asking me if I agreed or disagreed with certain statements about the brands’ history and engagement in the community, making matters worse by using a 6-point balance scale with no neutral point or Don’t Know option.

 

I like yogurt and have tried many brands, which makes me familiar with them until a certain point. However, my level of involvement is this category has been limited to looking at the label information to check for sugar, fat content, and all the extra fillers they put in yogurt. It has never been high enough to dedicate time to research the history and community engagement of any of the brands I have consumed. Many of them were not even in my top of mind list.  Since the scale didn’t have a Don’t Know option, I was forced to select answers that made feel like a liar, because I didn’t really know the answers, probably making the data useless.

 

SURVEY DESIGN REMEDY

 

1. If you are going to use a familiarity scale, make sure to define what each of the scale points means, and provide enough of them to capture different levels of familiarity.

 

2. A better solution yet is to ask specific questions to gauge familiarity with certain items to exclude unfamiliar ones, before you ask participants to agree or disagree with them.

 

3. An awareness would have been a more appropriate scale for some of the brand statements scale in the example above.If you use agreement scales, always add a Don’t Know option and a neutral point to the scale. Don’t assume consumers know enough about the subject or have a position formulated about it.

 

To read more about this, check: Is It Right to Include a Neutral Point in Rating Questions?

Only logged in users can leave comments.