Many social surveys are facing significant challenges, such as increasing nonresponse and survey costs. One approach to minimising costs is to shift from face to face (F2F) or telephone interviews to less costly web interviews. Research though, has shown that – while there may be benefits to web mode in terms of responses from individuals not sampled by in-person modes – generally non-response biases are minimised by offering both web and F2F or telephone modes. This has led some surveys to adopt designs in which sample members are first offered web mode, then non-respondents are followed up with F2F or telephone modes. However, with proportions of populations using the internet increasing markedly and people becoming less willing to welcome interviewers into their homes for interviews, in recent years the contributions of web and F2F or telephone modes to minimizing non-response biases that justify such a design may have changed. In this paper, we investigate this question in datasets from Understanding Society – the UK Household Longitudinal Study (UKHLS). We focus on the Innovation Panel component of the study, in which a subset of sample members has been offered web interviews with F2F or telephone follow-ups of non-respondents since 2012. We evaluate 1) the impact of following up web non-respondents on how well respondent datasets reflect sample datasets (dataset representativeness), and 2) how the impact of these follow-ups has changed over time. For each survey wave, we use Coefficients of Variation of response propensities (CV)s to quantify the representativeness of web only and web plus F2F or telephone respondents compared to the sample in terms of survey measured characteristics. In addition, we repeat our analyses using the UKHLS main survey dataset, whose more representative composition enables consideration of whether patterns differ for hard-to-reach population elements.

Co-authors

Dr Pablo Cabrera Alvarez; Professor Annette Jäckle, University of Essex;

Professor Gabrielle Durrant, University of Southampton