Simple Surveys: Response Retrieval Inspired by Recommendation Systems

Abstract

In the last decade, the use of simple rating and comparison surveys has proliferated on social and digital media platforms to fuel recommendations. These simple surveys and their extrapolation with machine learning algorithms like matrix factorization shed light on user preferences over large and growing pools of items, such as movies, songs and ads. Social scientists have a long history of measuring perceptions, preferences and opinions, often over smaller, discrete item sets with exhaustive rating or ranking surveys. This paper introduces simple surveys for social science application. We ran experiments to compare the predictive accuracy of both individual and aggregate comparative assessments using four types of simple surveys -pairwise comparisons and ratings on 2, 5 and continuous point scales in three distinct contexts -perceived Safety of Google Streetview Images, Likeability of Artwork, and Hilarity of Animal GIFs. Across contexts, we find that continuous scale ratings best predict individual assessments but consume the most time and cognitive effort. Binary choice surveys are quick and perform best to predict aggregate assessments, useful for collective decision tasks, but poorly predict personalized preferences, for which they are currently used by Netflix to recommend movies. Pairwise comparisons, by contrast, perform well to predict personal assessments, but poorly predict aggregate assessments despite being widely used to crowdsource ideas and collective preferences. We also demonstrate how findings from these surveys can be visualized in a low-dimensional space that reveals distinct respondent interpretations of questions asked in each context. We conclude by reflecting on differences between sparse, incomplete 'simple surveys' and their traditional survey counterparts in terms of efficiency, information elicited and settings in which knowing less about more may be critical for social science. is Professor at the Toyota Technological Institution of Chicago, and part time faculty in Computer Science and the Committee on Computational and Applied Mathematics at the University of Chicago. Srebro is interested in statistical and computational aspects of machine learning and their interaction. He has done theoretical work in statistical learning theory and in algorithms, devised novel learning models and optimization techniques, and has worked on applications in computational biology, text analysis, collaborative filtering and social science. Srebro obtained his PhD from the Massachusetts Institute of Technology in 2004.

A Digital Recommendation System for Personalized Learning to Enhance Online Education: A Review

“…Future work may involve further refinement of the system based on user feedback and the exploration of additional features to improve its effectiveness [56] ChatGPT Tool Users (Users No. are not mentioned) Application of ChatGPT tool to ameliorate education system, Explore the relationship between physical appearance and academic performance Explore additional factors influencing student performance, such as socioeconomic background Conduct longitudinal studies to assess the long-term effects of teaching mode on student outcomes [57] Machine Developed a recommendation system using machine learning algorithms to suggest tailored English teaching resources based on individual learning preferences and progress. Improved resource engagement by 25% compared to non-personalized approaches.…”

Social Computing Unhinged

“…High throughput surveys involve the use of crowdsourcing and deployment of simple information tasks at massive scales on digital platforms ranging from Wikipedia to XBox [43] . These often deploy active learning to make them adaptive and reduce sample size, focusing only on questions most relevant to respondents, information about which models are most uncertain, or both [44] . Finally, high throughput digital experiments are flourishing, facilitated by broadband that enables groups, teams and communities to interact with no perceivable latency.…”

Matrix Completion with Model-free Weighting

“…In its simplest form, a partial (noisy) observation of the target matrix is collected, and the goal is to impute the missing entries and sometimes also to de-noise the observed ones. There are various related applications in, e.g., bioinformatics Chi et al (2013), causal inference Athey et al (2018); Kallus et al (2018), collaborative filtering Rennie and Srebro (2005), computer vision Weinberger and Saul (2006), positioning Montanari and Oh (2010), survey imputation Davenport et al (2014); Zhang et al (2020); Sengupta et al (2021) and quantum state tomography Wang (2013); . Matrix completion has been popularized by the famous Netflix prize problem Bennett and Lanning (2007), in which a large matrix of movie ratings is partially observed.…”