In March 2020, the World Health Organization (WHO) issued a set of public guidelines for Coronavirus Disease 2019 (COVID-19) prevention measures that highlighted handwashing, physical distancing, and household cleaning. These health behaviors are severely compromised in parts of the world that lack secure water supplies, particularly in low- and middle-income countries (LMICs). We used empirical data gathered in 2017-2018 from 8,297 households in 29 sites across 23 LMICs to address the potential implications of water insecurity for COVID-19 prevention and response. These data demonstrate how household water insecurity presents many pathways for limiting personal and environmental hygiene and impeding physical distancing, exacerbating existing social and health vulnerabilities that can lead to more severe COVID-19 outcomes. In the four weeks prior to the survey, 45.9% of households in our sample either were unable to wash their hands or reported borrowing water from others, which may undermine hygiene and physical distancing. Further, 70.9% of households experienced one or more water-related problems that potentially undermine COVID-19 control strategies or disease treatment, including insufficient water for bathing, laundering, or taking medication; drinking unsafe water; going to sleep thirsty; or having little-to-no drinking water. These findings help identify where water provision is most relevant to managing COVID-19 spread and outcomes.