Writing a survey can sometimes be challenging. But anything worth doing is challenging. We’ve heard that too, so no worries.
We love a good challenge, especially when it has to do with—take a guess—yep, surveys.
Back to (page) basics
One of the first questions that may come to mind for you is simply, how many pages should you have in a survey?
Sure, you can go ahead and include all questions on one single page. But don’t forget, you can also choose to place each question on its own page.
In the survey research world, the one question per page survey is called paging design while the multiple questions per page survey is referred to as scrolling design.
Researchers started to examine this very important survey design feature over a decade ago. They started by testing the paging vs. scrolling designs using PC web surveys and didn’t find much difference between these two formats in terms of survey completion rate and break-off rate.
That means respondents were equally likely to complete the survey whether it involved paging or scrolling. However, some studies found that respondents were more likely to skip questions when the survey required scrolling. This is possibly due to survey fatigue when too many questions were presented on the same web page.
The pros and cons of paging vs. scrolling
The rapid growth of mobile web surveys add another layer of complexity for survey designs.
On the one hand, given the smaller screen size of smartphones, presenting one question at a time through a paging design might be a good idea. On the other hand, a paging design requires the respondents to click the Next button every time they finish a question. This can take quite a lot of effort, especially when the surveys have too many questions and pages.
Also, the internet connectivity for smartphones is usually not as reliable as for computers. That of course can help to slow respondents’ ability to complete surveys and inadvertently cause survey break-off.
One study published in 2014 by researchers from the US and Russia confirmed some of these hypotheses. The study compared the two survey designs that were taken by mobile phone respondents only. So what’d they learn? People took much longer to complete a survey when it had a paging design versus a scrolling design.
Also, when presenting questions page-by-page, instead of on a single page, respondents experienced more technical difficulties and expressed lower satisfaction with the survey. Interestingly, the numbers of questions skipped by respondents are similar between the two survey design formats.
4 pieces of survey design advice
1. Is your survey super short? Are you expecting most people who take it to be on a mobile device? Then include all questions on one page.
2. Is your survey long (but hopefully not too long)? Spread questions out onto a few pages. But! Don’t show only one question per page—you’ll end up with way too many pages.
3. Does your survey have skip logic, randomization, or question and answer piping? You’ll definitely need more than one page to fit your questions.
4. Does your survey have more than one page? That’s okay. Why? Because you can be assured you’ll capture at least some data.
If people drop out of the survey halfway through and all the questions are shown on the same page, no data will be captured. However, if the break-off happens on the third page, the data provided on the first two pages are preserved and may still be useful.
Need more proof, or have questions for our Survey Scientist? Let him know in the Comments below!
