This year we reprised our election project to predict the outcomes of the Virginia and New Jersey gubernatorial elections. We are proud to report that we correctly predicted the winners, and were within a few percentage points of the actual vote totals.
And the best part is we did all of this using UNWEIGHTED, RAW data. How did we do so well? Part of the key to our success was the high quality of our sampling process.
You’ve probably heard that “sampling” is something that you should do when sending out a survey. But what exactly that might mean or why you should care might be a little fuzzy.
Well, we’re here to illustrate how vitally important good sampling technique is to the success of your survey using some data from our 2013 election project. Our take-home point is that how you hunt down the people you ask really does matter. It’s not just what you ask and how you ask it, but WHO you’re asking!
In sampling you have 3 important factors to consider:
- Population: The people you WANT to contact
- Sampling frame: The people you CAN contact
- Sample: The people you DO contact
In the context of our election project here is how those 3 factors play out:
- Population: Eligible voters who show up to vote in a state
- Sampling frame: Anyone in that state who takes any SurveyMonkey survey
- Sample: Any of those survey takers who take our election survey
Here’s some maps of those same sampling variables from our 2013 Election Project:
(A darker color means more people came from that part of the state, a lighter color means fewer did.)
So, why is sampling so important? Well, if we have a sampling frame that is representative of our population, this gives us more leeway with our actual sample. If we don’t, then we’ll have to work harder to make sure the sample is representative of our population.
The high quality of our sampling frame in our election project (indicating that it does a good job of representing the population) means that our actual sample can deviate a little more from the actual population without it being a problem. When you have a more representative pool (in this case, anyone completing a SurveyMonkey survey) to draw people from than say, people who pick up their phones during dinner, your chances of uncovering the truth are a lot higher.
Well, there you have it! Now you know why sampling matters AND the secret to our success in the 2013 Election Project. Smart sampling.
So, on your next survey, before you hit send – think about WHO you’re sending it to. It’s the best way to make sure you get not just answers, but the RIGHT answers.
