Asking questions about past behaviors in a survey is quite common, but also problematic. The assumption is that respondents can give an accurate account of what they did, how often they did it or how much time, energy and money they spent doing it. The fact is that our memory fails more than we want to admit. Respondents, often and unintentionally make memory-based errors which lead to over- or understatements of their actions.

Factors that affect memory in the context of a survey include:

• The type of event is being asked about
• Feelings about and meaning of the event
• How often an event has happened
• How long ago the event happened
• How the question is formulated

All these factors impact our ability to recall a particular event while participating in a survey and/or misplace an event in time, also known as telescoping effect (recent events are perceived as being remote and remote events as being recent).

Research has found partial explanation to this phenomenon in the way our memories are organized. Some are stored as episodic memories. These are concrete personal events determined by a certain time and place. Others become part of our semantic memory, which includes our abstract knowledge about the world, not necessarily tied to specific events.

It turns out that the use of one type of memory or the other is influenced by the time frame of the question and frequency of events. For example, studies have indicated that while estimating frequency in the occurrence of an event, we tend to use our episodic memory for events that are rare (e.g. car accidents) or are recent (e.g. grocery shopping), while we often use our semantic memory when the reference period is long (e.g. past 12 months vs. past 3 months), as we have a harder time to keep the events separate as unique memories over time.

Depending on the type of event and time frame, the telescoping effect can be stronger going forward (distant events perceived as recent) or backwards (recent events perceived as distant). For example, for rare and remote events, we are more likely to misplace them in a more recent time (telescoping forward), while the opposit happens if they are recent.

For obvious reasons this has big implications for the quality of the data collected via surveys, particularly when past behaviors within a particular time frame are used for sample selection. This also has an effect on response accuracy for many questions related to the particular event or behavior that it is being studied.

For example, in many cases when a respondent is asked questions about the last time he or she made a purchase on a particular category in the last 12 months, he or she maybe just giving answers about his or her average behavior over a vaguely defined period of time relying more on his or her semantic memory than on the episodic memory of the last occasion if this is a frequent behavior. The opposite may also happens, when we ask about an “average” or “typical” behavior and a respondent bases his or her answers on the most recent event which is fresh in his or her memory, generalizing one experience to all previous experiences, which or may not be similar.

What can we do to minimize memory-based errors in questions about past behaviors?

• Keep the reference period for the event of interest closer to the time in which the data is collected. You are likely to give more accurate responses about events that happened in the last 7 days than in the past 12 months.
• Refer to meaningful events to anchor the time frame. This can be dates and specific events to a particular time period (e.g. before or after 9/11).
• Provide clues that can be associated with the behavior of interest (e.g. How many times do you drink beer in a week? Think of occasions in which you may drink beer: party, after work, meals, watching TV, with friends, etc.)
• Use “warming-up” questions to trigger memories related to the event of interest (e.g. Do you like beer? What type? How many?)
• Refer to previous answers (e.g. You indicated you have been to the Dr. this month. How many times have you visited your primary Doctor?)
• Use records if available (e.g. Our records indicated you called our customer service on mm/dd. How long did you have to wait on the phone?)

Some of these measures are easier said than done. Survey length, sample incidence rate, survey tool limitations, lack of information, and other research priorities can get in the way of the appropriate survey design including questions about past behaviors. When you face these obstacles, do not forget the implications for data quality that not taking measures to minimize memory-based error have and persist in finding a way to do it, otherwise you may end up wasting time and money in gathering misleading data.