Introducing web in a refreshment sample of the Swiss Household Panel: Main findings from a pilot study

This report summarizes the main findings of a two-wave pilot for the SHP conducted in 2017 and 2018, designed to test the consequences of including web as a main mode in the data collection in terms of data quality and comparability between modes. The pilot had two main goals: to test the feasibility of starting a new sample using web and the inclusion of web on the individual level only versus on both the household and the individual level. The study compared three design groups: two treatment groups with web as main survey modes and one control group with the usual design of the SHP and telephone as main mode. We analysed overall response rates, sample composition and representativeness, item nonresponse and measurement differences between the telephone and web mode. We found that the response rates in the first wave, hence at recruitment into the panel, were highest in the usual design and lowest in a full web design, whereas modes mattered less in the second wave, once households had been recruited into the panel already. Switching modes between the household and the individual level decreased individual response rates, whereas switching from telephone in Wave 1 to web in Wave 2 on the household level did not have any negative impact on response rates. We assessed the sample composition by comparing characteristics of respondents with those of the full sample. We found that most of the bias emerged in the first wave, with selection effects getting smaller in the second wave. Some of the sociodemographic characteristics were biased in all designs, whereas others were more specific for each design, suggesting that the choice of mode affects the composition of the sample recruited into the panel in the first wave. For both waves, item nonresponse was considerably higher in the web mode. Although for many variables, we found no indication of measurement differences by mode, a few clear exceptions emerged that suggest a social desirability effect in telephone interviews: web respondents were more likely to report health problems and negative emotions. Also, satisfaction scores were consistently lower in the web mode.