Greetings, dear customers! The brand new Survey Research team here at SurveyMonkey is making its official debut on the blog today and we couldn’t be more excited to share best practices and tips with all of you. Let’s jump right in, shall we?
Using SurveyMonkey Audience–a powerful tool for targeting a specific demographic–we recently launched a political survey in seven swing states asking people their opinions on the Affordable Care Act aka “Obamacare.” Each of these states was identified by the nonpartisan online newsletter, Cook Political Report, as having competitive statewide races in the 2014 election year.
We wanted to dig deeper into one of the open-ended questions we asked on the survey, and let you in on how you can make sense of those responses and ensure the quality of your data. After all, smart data leads to even smarter decisions.
Here’s our open-ended question: “In a few words how would you describe your feelings about the health care law known as Obamacare?” And a peek at some of the responses we received:
One of the great things about open-ended questions? People aren’t limited to a predetermined set of possible answer choices so you end up collecting a rich pool of genuine opinions from folks on your survey topic. However, they also present an analytical challenge–just how do you make sense of all these unique answers?
For starters, thanks to our partnership with NVivo you can easily import your SurveyMonkey data into the NVivo platform to analyze the text. Alternatively, you can do a basic analysis right inside our Analyze tool and categorize the responses to provide not only a detailed picture of what people’s opinions are in their own words, but also to know how many people feel that way. To use the Categorize feature, just tick the box next to each response in order to place it into a category. So! Ready to become an open-ended expert?
Five tips to get you started
- Read through a couple of responses to get a sense of what folks are saying. As always, know your data. By glancing at the images of the responses above, you’ve already done the first step.
- Map out a few general categories to put each of the responses in. As you read the responses above, you can already see that folks are pretty much divided into three camps–those that like the law, those that hate it, and those who don’t have an opinion either way–so we created three categories: positive opinions about the law, negative opinions, and neutral opinions.
- Create sub-categories underneath your general ones to provide even richer detail. As you move along and put responses in their respective buckets (positive, negative, or neutral in this example), you’ll see some reoccurring themes popping up within each general category. Group and tag those themes together to create new sub-categories. For example, we saw lots of respondents saying that the law was “socialistic,” so we created a sub-category for that underneath “negative feelings.” This extra information is totally optional, so if you don’t need this level of detail, feel free to skip on down to step 4.
- Double check and re-categorize. Go back and re-read responses to make sure they properly fit in the categories that you’ve assigned them to. If you’ve added sub-categories along the way, you’ll be able to tag previous responses to the new sub-categories on this second look-through. Also, don’t be afraid to assign multiple categories to one response since often open-ended comments cover more than one category. But note that since each response can be assigned to multiple categories, your percentages may not add up to 100.
- Put a number on it! Congrats, you’re done! Click over to the My Categories tab in your question summary to see the percentages. After finishing our open-end response analysis in our political survey, we found that among those we surveyed–35% had positive things to say about the law, 55% had negative comments, and 10% had neutral comments, similar to national opinion on the law.
Since we added extra sub-categories, we were able to tell a more detailed story than if we had only looked at positive, negative and neutral as categories. For example, 10% of folks mentioned that the law was a good idea, but that due to the technical problems with the roll-out of the Obamacare website, the law needed work.
That’s it for today, everyone. We hope these tips help out the next time you want to make sense of your open-ended responses and don’t be shy, let us know your thoughts!
