Online questionnaire design: standard matrix or scroll matrix?

As a research agency with a strong academic background and commitment, it is important to us to contribute to the development of our field.
Completing standard questionnaires is not always a fun thing to do for respondents, with negative consequences for the response and the quality of the answers. The traditional matrix question is under attack and is characterised as demotivating. An attractive alternative is the scroll matrix question. The principle of the scroll matrix is that only one question is visible at a time. Once an answer is filled out, the following statement appears automatically. In order to examine the effectiveness of the scroll matrix compared to the traditional matrix, Flycatcher has carried out a large-scale online experiment in collaboration with the Utrecht University among the Flycatcher panel.

The experiment showed that when using the scroll matrix fewer respondents drop out, and that the data quality is better. The response bias is lower than in the standard matrix. This applies to all types of examined response bias: straightlining, limited response range, and tendencies for extreme responses and neutral responses. The type of question had no influence on the inter-item-correlation and the reliability of indices. Finally, it showed that panel members experienced filling out the scroll matrix as more pleasant and that in their perception it also seemed to go faster.