Non-probability sampling
The simplest way to get results for everyday surveys.
Running a large-scale survey can be a tricky task. As much as you need your results to represent the entire population, it’s hard to give everyone you’d like to hear from the chance to be surveyed.
One real-world solution is to use non-probability sampling, which SurveyMonkey knows a thing or two about. With more than half a million people available to take surveys through our Audience panel at any time, SurveyMonkey has the largest non-probability sample in the US.
Non-probability sampling selects a group of respondents from a larger population, knowing full well that some members of the population have zero chance of being surveyed. This is not allowed in probability sampling, which requires everyone in the population to have a non-zero chance of being selected.
Not everyone gets a shot at being selected
Whether you’re using a panel like SurveyMonkey Audience or any other type of non-probability sampling design, the way you select respondents will always knowingly leave out some members of the population.
Sometimes these exclusions are obvious, like when people can opt in or out of responding to you. For example, you might ask your customers to provide their emails so that they can participate in a customer feedback survey. Some will probably decline, which means they have no chance of being selected into your survey sample.
Other times, these exclusions are subtler. Let’s say you plan to survey the first 100 people who walk into your store on a given day. This may seem like a random probability sampling design, but consider this: there’s probably a difference between the type of people who can come to your store in the morning versus those who have to visit later.
If your store opens at 9 am, maybe your first customers of the day are less likely to be employed than those who are coming in at 7 or 8 at night. Since some part of the population has no chance of being among those first 100 customers of the day, your results may be biased—so you’re actually using non-probability sampling here.
Common non-probability sampling strategies
Here are some non-probability sampling designs that are used regularly, even if they’re not well-suited to all surveys:
- Quota sampling. Set specific targets for the number of people that you want to survey (e.g., 50 men and 50 women), and then stop after you’ve reached each target. Quota sampling ensures that you get at least some respondents from all the subpopulations you’re interested in, even though this still isn’t a true probability sample.
- Convenience sampling. Ask only people you know or people who are readily available to complete your survey. This is fine if you’re just doing a survey for fun (e.g., asking 100 people on the streets of New York what they think about a celebrity running for president), but if you’re trying to produce widely applicable results, you’ll need to use a method that’s more scientific.
- Snowball sampling. Ask people who are already participating in your survey to recruit other survey participants that they know. Snowball sampling is best for surveys targeting specific groups that are hard to find or reach, like undocumented immigrants or people with rare health problems. In this case, you can assume that the population you’re interested in is relatively homogeneous, and you don’t have to worry as much about having a representative sample.
If you're planning to use non-probability sampling methods, you'll need practical tools to select your sample. Learn how to create a random sample using Excel to efficiently select participants from your available population.
The pros and cons of non-probability sampling
Probability sampling is favored by statisticians, but for people conducting surveys in the real world, non-probability sampling is more practical. If done well, non-probability sampling can give you the same (or better) high-quality data you would expect from a true probability sample.
Most surveys are targeted at a very specific population and don’t need to ensure the same diversity and representation provided by probability sampling. If you’re doing market research on mothers of young children, you don’t need a probability sample that includes men, people without children, or people with adult children.
Even when a non-probability sample doesn’t perfectly overlap with your population of interest, there are still plenty of advantages to sticking with it.
Getting responses with non-probability sampling is usually faster and cheaper than getting them with probability sampling, because sample members are more motivated to respond than people who are randomly contacted. People selected from a mailing list, for example, are probably more loyal to a company than people chosen from outside of it.
The biggest challenge of non-probability sampling is recreating the same kind of non-biased results that probability sampling gives you.
Always be careful that the way you’re recruiting respondents isn’t distorting your data. Some online panels pay their respondents, which can lead to bias from “professional” survey-takers who answer just for the money and are not providing accurate information.
When you’re doing a non-probability survey, be sure to think through potential sources of bias. It’s not always easy to predict what will bias your results, but starting with a diverse group of respondents with characteristics that match your population of interest is essential. This will not only give you data that is just as accurate as probability sampling, but it will be much more cost- and time-effective, too.
Discover more resources
Toolkits directory
Discover our toolkits, designed to help you leverage feedback in your role or industry.
Survey with file upload: How-to, examples, and tips
Forms gather vital info, but sometimes you need to conduct a survey with a file upload option. Here’s how to create a survey with file upload.
How to find survey respondents in 5 steps (with examples)
Get the best data from your survey. Learn how to find survey respondents people with these tools and tips from our survey research experts.
