This publication is part of the RTI Press Research Report series. Occasional Papers are scholarly essays on policy, methods, or other topics relevant to RTI areas of research or technical focus.

Reduction in nonresponse bias has been a key focus in responsive and adaptive survey designs, through multiple phases of data collection, each defined by a different protocol, and targeting interventions to a subset of sample elements. Key in this approach is the identification of nonrespondents who, if interviewed, can reduce nonresponse bias in survey estimates. From a design perspective, we need to identify an appropriate model to select targeted cases, in addition to an effective intervention (change in protocol). From an evaluation perspective, we need to compare estimates to a control condition that is often omitted from study designs, in addition to the need for benchmark estimates for key survey measures to provide estimates of nonresponse bias. We introduced a bias propensity approach for the selection of sample members to reduce nonresponse bias. Unlike a response propensity approach in which the objective is to maximize the prediction of nonresponse, this new approach deliberately excludes strong predictors of nonresponse that are uncorrelated with survey measures and uses covariates that are of substantive interest to the study. We also devised an analytic approach to simulate which sample members would have responded in a control condition. This study also provided a rare opportunity to estimate nonresponse bias, using rich sampling frame information, prior round survey data, and data from extensive nonresponse follow-up. The bias propensity model yielded reasonable fit despite the exclusion of the strongest predictors of nonresponse. The intervention was found to be effective in increasing participation among identified sample members. On average, the responsive and adaptive survey design reduced nonresponse bias by more than one-quarter—almost one percentage point—regardless of the choice of benchmark estimates. Effort under the control condition did not reduce nonresponse bias. While results are strongly encouraging, we argue for replication with varied populations and methods.

The resources needed to conduct high-quality and large-scale survey research are often beyond the reach of institutional researchers and higher education analysts. However, the National Center for Education Statistics (NCES) provides many national student surveys that researchers can utilize. We outline four NCES studies-the High School Longitudinal Study of 2009, the National Postsecondary Student Aid Study, the Beginning Postsecondary Students Longitudinal Study, and Baccalaureate and Beyond Longitudinal Study-and describe their samples and research topics that can be explored with each. Using practitioner focused case studies as examples, we then describe three ways that researchers can access and utilize these data. We also provide an appendix with code for importing data, using weights, and estimating variance in R, Stata, SAS, and SPSS.Collecting high-quality student data can be a daunting or expensive task for many institutional researchers or higher education analysts. This volume highlights the many resources needed for the various components of survey data collection, such as: generating a statistical sample, maximizing response rates, and correcting for non-response via weighting. The effort required to reach the "gold standard" in survey design often presents an unreasonable burden for those trying to engage in large-scale research projects, particularly those with scarce resources and demanding timelines. Fortunately, higher education researchers have more opportunities to access student data from secondary sources than ever before. One of the most well-known providers of large-scale student survey data is the National Center for Education Statistics (NCES). NCES is one of several federal statistical agencies housed within the U.S. Department of Education. It is entrusted with data collection, analysis, and reporting, and its products are available to the public for institutions and researchers to access. As other NEW DIRECTIONS FOR INSTITUTIONAL RESEARCH, no. 181

Surveys often require monitoring during data collection to ensure progress in meeting goals or to evaluate the interim results of an embedded experiment. Under complex designs, the amount of data available to monitor may be overwhelming and the production of reports and charts can be costly and time consuming. This is especially true in the case of longitudinal surveys, where data may originate from multiple waves. Other such complex scenarios include adaptive and responsive designs, which were developed to act on the results of such monitoring to implement prespecified options or alternatives in protocols. This paper discusses the development of an interactive web-based data visualization tool, the Adaptive Total Design (ATD) Dashboard, which we designed to provide a wide array of survey staff with the information needed to monitor data collection daily. The dashboard was built using the R programming language and Shiny framework and provides users with a wide range of functionality to quickly assess trends. We present the structure of the data used to populate the dashboard, its design, and the process for hosting it on the web. Furthermore, we provide guidance on graphic design, data taxonomy, and software decisions that can help guide others in the process of developing their own data collection monitoring systems. To illustrate the benefits of the dashboard, we present examples from the National Longitudinal Study of Adolescent to Adult Health (Add Health). We also discuss features of the dashboard to be developed for future waves of Add Health.