A survey is like a snapshot: From one survey, you can only draw conclusions about a single time, place, and group of people. And often, that’s all you need to know.
But sometimes, you also want to understand how the people you surveyed are changing. In that case, one single survey, or snapshot, is not enough—so we need to repeat surveys in order to track and understand trends over time.
If you want to see how people are changing, there are two ways you can do so. First is benchmarking, which means that you are asking different groups of people the same question over time to see how views change. The second way, and the focus of this post, is a longitudinal survey.
When you run a longitudinal study or survey, you’re essentially following the same group of respondents over a long period of time, for weeks, months, or even years.
This differs from a cross-sectional survey, which is a fancy way of saying that each of your survey respondents only completes the survey once, but you might conduct the survey multiple times to collect some benchmarking data. (That’s your snapshot survey.)
But why would you want to run a longitudinal study? Well, just like this study that began in 1968 and still runs today, you might want to monitor changes over the course of your respondents’ lifetimes to be able to draw conclusions from a (very consistent!) group of respondents.
How repeating surveys will help you make smarter choices
While you may not be in the market for running a decades-long study anytime soon, you can benefit from repeating surveys and tracking changes in your respondents’ attitudes and behaviors over time. (By the way—when you survey the same people time and time again, you’re running what’s also called a panel survey.)
For example, say you’re an online marketer who wants to know how your readers will react to a new email newsletter design.
Increase the power of your brand
You can’t improve what you don’t measure. Start tracking your key brand metrics today with SurveyMonkey Audience.
Instead of just sending your readers a survey after you’ve changed your design, send them a survey asking them what they like (and don’t like) about the current design (i.e., create a concept test). You can even use their feedback to inform your newest design!
Then, send them a follow-up survey after you’ve sent them the newly designed newsletter. Because you’re surveying the same people, you can compare their attitudes and opinions of the first design against how they react to the second design and smaller changes will be statistically significant. If you decided to do two cross-sectional surveys with different groups of people, you would need to see a larger change in order to see a significant difference.
If you make more changes to the design based on your readers’ feedback, you can continue to refine your design over time and make sure that satisfaction ratings don’t dip below the initial satisfaction ratings with the first design.
An alternative to the longitudinal panel survey
Repeating surveys with the same panel works well when you’re tracking changes in your respondents’ attitudes and behaviors, but sometimes, you’re unable to survey the exact same people time and time again.
In this case, even if you are sending your newsletter out to the same people, you may not be able to capture the same opinions. People may unsubscribe from your newsletter, and newer readers may come along.
That’s when you conduct something called a rotating panel survey. All you need to do is gradually rotate a portion of your initial sample out of the panel survey and supplement them with new readers. (In this case, you could easily track who from your email list is filling out your surveys.)
This way, your survey will provide a good estimate of the opinions of your entire readership, old or new, while at the same time capturing the changing opinions of the same group of people.
3 tips for creating surveys for a longitudinal study
Here are three things you need to keep in mind when creating your longitudinal study:
- It’s better to keep the questions identical for all the surveys. Research has shown that changing the way a question is asked can result in substantially different answers, even from the same people. So to examine trends with the most accuracy, you should try your best to ask the same questions in every survey you repeat.
- When planning a panel survey, remember that not everyone who responded to the first survey will respond to the second survey. So if you want to do a panel survey where you survey the same group of people 3 times and want at least 1000 respondents for your third survey, you’ll need to survey more than 1000 respondents in your first survey to account for people who probably won’t respond to your second or third survey.
- You need to decide how frequently you want to run your survey. If you conduct surveys too frequently, you could waste your resources and time, since not enough time may have passed for any change to occur. However, if you conduct surveys too infrequently, more people may drop out of your panel survey because they are sick of taking too many surveys, which means that you’ll have fewer respondents in your later surveys.
When you think about it, the applications for a longitudinal study are endless. You can see whether your new ad actually influenced people purchasing your product or follow up with product purchasers to see if they’ve enjoyed using your product.
Even though one dataset can shed light on a single occurrence, the context that comes from repeating surveys over time will help you make informed decisions and improvements.
Questions, comments for Mingnan? Let him know below!
