Measures of cognitive, noncognitive, and technical skills are increasingly used in developing country surveys, but have mostly been validated in high-income countries. We use a survey experiment in Western Kenya to test the reliability and validity of commonly used skills measures. Cognitive skills measures are found to be reliable and internally consistent, technical skills are very noisy, and measurement error in noncognitive skills is found to be non-classical. Addressing both random and systematic measurement error using common psychometric practices and repeated measures leads to some improvements and clearer predictions, though concerns remain. These findings hold for a replication in Colombia.
JEL codes: O12, O13, O15The data and programs for full replication are available online at https://www.laajaj.com/research 1 Jack Pfeiffer provided invaluable research assistance and programming skills, field support, tenacity and ideas throughout the project and we gratefully acknowledge all his contributions to this project. provided excellent research assistance. Data collection was organized through IPA Kenya and S.E.I. in Colombia and we acknowledge the excellent field teams for key contributions during piloting and translation and for all efforts during the data collection. We are indebted to Chris Soto for many suggestions and insights from the psychometric literature. This open access article is distributed under the terms of the CC-BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0) and is freely available online at: http://jhr.uwpress.org 2 Hence we broadly follow the distinction of Heckman and Kautz (2012) who distinguish between cognitive abilities, personality traits, and other acquired skills. Jones and Kondylis (2018) is a recent example using a measure of acquired agricultural skills, in which measurement error is being discussed as potential reason for lack of impact. 3 Measures of risk aversion and time preferences, which have a longer history of use in developing country surveys, have received more scrutiny. Chuang and Schechter (2015) review the evidence, including their stability over time. 4 While explanatory factor analysis is used elsewhere in the economics literature on skills Heckman, Stixrud and Urzua, 2006), we also build on insights from the psychometrics literature, such as for the corrections for acquiescence bias (the tendency to agree even when statements are contradictory, or "yea-saying"). 5 Following McKenzie (2012), in order to reduce the noise in the outcome variables, the measures of yield and practices are obtained from the average over the four seasons that followed the collection of skills data.