Does A Large Sample Size Guarantee A Representative Sample?

Twitter Facebook

by Michaela Mora
Follow Me on Twitter Here

by Michaela Mora Follow me on Twitter

Posted on May 13, 2010

Representative Sample

I often get asked "What sample size do I need to get a representative sample?" The problem is that this question is not formulated correctly. 

Sample size and representativeness are two related, but different issues. The sheer size of a sample is not a guarantee of its ability to accurately represent a target population. Large unrepresentative samples can perform as badly as small unrepresentative samples.

A survey sample's ability to represent a population has to do with the sampling frame; that is the list from which the sample is selected. When some parts of the target population are not included in the sampled population, we are faced with selection bias, which prevent us from claiming that the sample is representative of the target population. Selection bias can occur in different ways:

  • Convenience sample: This includes respondents who are easier to select or who are most likely to respond. This sample will not be representative of harder-to-select individuals. Samples from online panels are a good example of convenience samples. These panels are composed by individuals who have expressed interest in participating in surveys, leaving out individuals who may be part of the target population, but are not available for interviewing through the panel.
  • Undercoverage: This happens when we fail to include all the target population in the sampling frame. Many online panels work hard at avoiding undercoverage bias, but the fact remains that certain demographics are underrepresented. For example, it is difficult to field online studies targeted at the total Hispanic population in the US without using a hybrid data collection approach that allows us to reach unacculturated Hispanics, who are usually underrepresented in most online panels. Coverage bias is also found in phone surveys that use telephone list sampling frames that exclude households without landline access. As more households substitute cell phones for their landlines, obtaining representative samples of certain demographic groups will soon be difficult without including cell phone lists in the sampling frame.
  • Nonresponse: Selection bias also takes place when we fail to obtain responses from all respondents in the selected sample. Nonrespondents tend to differ from respondents, so their absence in the final sample makes it difficult to generalize the results to the overall target population. This is why the design of a survey is far more important than the absolute sample size to get a representative sample of the target population.
  • Judgment sample: This is a sample selected based on "representative" criteria based on prior knowledge of the topic or target population. An example would be a study looking for a sample of teenagers, and trying to intercept them at a cross-section near a high school.
  • Misspecification of target population: This happens when we use intentionally or unintentionally screening criteria that leave out important subgroups of the population.
  • Poor data collection quality: This can introduce selection bias when there are poor quality controls to ensure that we interview the designated members of the sample. An example of this include allowing whoever is available in the household to take the survey instead of the intended member based on certain screening criteria.

So when it comes to getting a representative sample, sample source is more important than sample size. If you want a representative sample of a particular population, you need to ensure that:

  1. The sample source includes the whole target population
  2. The selected data collection method (online, phone, paper, in person) can reach individuals, with characteristics typical of those possessed by the population of interest
  3. The screening criteria truly reflect the target population
  4. You can minimize nonresponse bias with good survey design, incentives and the appropriate contact method
  5. There are quality controls in place during the data collection process to guarantee that designated members of the sample are reached.

For help on sample size calculation use our Sample Size and Margin of Error Calculators.



Tags: , , ,



5 Comments »
  1. Convenience sample explains about those people who have interest in participating survey. what do we call others.

    Comment by Sample Questionnaires — June 7, 2010 @ 1:16 am

  2. When we select a sample from a sample frame where we know the probability of someone being chosen, we call that a random sample. They are given a chance to participate regardeless whether they want to participate or not.

    Comment by Michaela Mora — June 7, 2010 @ 8:18 am

  3. Actually, convenience sample refers to how people are chosen to participate, (i.e., everyone who shows up at a clinic on a certain day that is chosen for data collection), not whether they opted to participate.

    Comment by CJ — June 23, 2010 @ 2:06 pm

  4. There is an opt-in component when we talk about online panels, not necesarily particular studies. When we use online panels, we get samples from people that happen to be in the panel the same way you get those that happen to show up at the clinic. Panel samples are non-probability samples. Not all members of the population have the same chance of being chosen, but only those who are part of the panel.

    Comment by Michaela Mora — June 23, 2010 @ 2:23 pm

  5. Indeed it’s not so hard to calculate a representive sample, the hard part is to conduct the survey on this sample cause there are always people who don’t to part of a survey

    Comment by Example Questionnaire — August 3, 2011 @ 10:18 am

RSS feed for comments on this post. TrackBack URL

Leave a comment

 

Subscribe
To Our Blog
Read market research articles with zero fluff!

Our Clients Say...

Red Mango needed first class consumer insight on a startup company budget. And we needed it fast. Relevant Insights was able to handle study design, execution, field work, and analysis in a way that met all of our needs.

Jim Notarnicola, CMO
Red Mango USA
Certifications
  • Minority Business Enterprise (MBE), National Minority Supplier Development Council (NMSDC)

  • Women Business Enterprise (WBE), Women's Business Enterprise National Council (WBENC)

  • Historically Underutilized Business (HUB), State of Texas

  • Disadvantaged Business Enterprise (DBE), North Central Texas Regional Certification Agency (NCTRCA)