Assessment of household appliance surveys collected with Amazon Mechanical Turk

Abstract

Energy researchers need data on residential appliances to make effective recommendations for reducing energy consumption. For some products, however, traditional data sources do not have sufficient detail. Online surveys can provide a less expensive alternative for data collection, but the accuracy of these surveys is still unclear. Here, we compare the results of Amazon Mechanical Turk (AMT) online surveys of refrigerators, freezers, televisions, and ceiling fans to the nationwide Residential Energy Consumption Survey (RECS) deployed by the U.S. Energy Information Administration (EIA). To account for differences in demographic distributions between the online survey results and the general population, we weighted the results using standard cell weighting and raking techniques, as well as a combination of these, termed "hybrid." The weighted results gave a distribution of product ownership that was reasonably close to RECS, albeit with small, statistically significant differences in some cases. The cell weighting method provided a slightly better agreement with RECS than the other two approaches. We recommend AMT online surveys as an efficient and cost-effective way of gathering in-home use data on appliances that are not adequately covered by existing data sources.2

The Influence of Colorants, Flavorants and Product Identity on Perceptions of Naturalness

“…Some respondents are inattentive and may answer questions without thinking or simply check boxes without reading the question [29,30]. Because of this, a fake or “cheater” question (e.g., [31]) was included in the survey. Participants who reported consuming ‘Live worms’ or ‘Pickled chicken’ in the past week were excluded from the analysis as these would be extremely unusual foods for the three countries tested.…”

Survey of Hearth Products in U.S. Homes

“…In Yang et al 2015, the cell weighting method was found to give better overall performance than the raking method in regards to reducing the mean and maximum absolute error among several product ownership measures. However, a potential disadvantage of cell weighting is that it can introduce great instability in weighting adjustment when the sample size of a number of cells is small.…”

“…The RECS hearth products subsample was extracted as detailed in Table 2.5, which shows the criteria, the RECS variables used for that criteria, the number of U.S. households identified by that criteria, and the representative households in the sample. The effectiveness of different survey weighting techniques were assessed by Yang et al (2015). This study specifically compared the cell weighting and raking weighting methods, as defined by a prior study (Kalton and Flores-Cervantes, 2003).…”

Trends in penetration and ownership of household appliances

“…The effects of the increase in usage of domestic computers in UK were evaluated by Terry and Palmer [12]. Finally, in USA Yang et al [13] recognized on-line surveys as a cheap and efficient way to collect data on residential appliances. Other authors intended to relate moving home [14], social habits [15], demographic changes [16], and income structure [17] to the ownership of appliances in developed countries such as United Kingdom, United States, and Japan.…”