The spread of SARS-CoV-2, like that of many other pathogens, is governed by heterogeneity. "Superspreading," or "over-dispersion," is an important factor in transmission, yet it is hard to quantify. Estimates from contact tracing data are prone to potential biases due to the increased likelihood of detecting large clusters of cases, and may reflect variation in contact behavior more than biological heterogeneity. In contrast, the average number of secondary infections per contact is routinely estimated from household surveys, and these studies can minimize biases by testing all members of a household. However, the models used to analyze household transmission data typically assume that infectiousness and susceptibility are the same for all individuals or vary only with predetermined traits such as age. Here we develop and apply a combined forward simulation and inference method to quantify the degree of inter-individual variation in both infectiousness and susceptibility from observations of the distribution of infections in household surveys. First, analyzing simulated data, we show our method can reliably ascertain the presence, type, and amount of these heterogeneities with data from a sufficiently large sample of households. We then analyze a collection of household studies of COVID-19 from diverse settings around the world, and find strong evidence for large heterogeneity in both the infectiousness and susceptibility of individuals. Our results also provide a framework to improve the design of studies to evaluate household interventions in the presence of realistic heterogeneity between individuals.