This article (a) provides a general review of interobserver reliability (IOR) and (b) describes our method for assessing IOR for items and amounts consumed during school meals for a series of studies regarding the accuracy of fourth-grade children's dietary recalls validated with direct observation of school meals. A widely used validation method for dietary assessment is direct observation of meals. Although many studies utilize several people to conduct direct observations, few published studies indicate whether IOR was assessed. Assessment of IOR is necessary to determine that the information collected does not depend on who conducted the observation. Two strengths of our method for assessing IOR are that IOR was assessed regularly throughout the data collection period and that IOR was assessed for foods at the item and amount level instead of at the nutrient level. Adequate agreement among observers is essential to the reasoning behind using observation as a validation tool. Readers are encouraged to question the results of studies that fail to mention and/or to include the results for assessment of IOR when multiple people have conducted observations. Address correspondence to: Suzanne Domel Baxter, PhD, RD, FADA, Arnold School of Public Health, Department of Epidemiology and Biostatistics, Center for Research in Nutrition and Health Disparities, University of South Carolina, 220 Stoneridge Dr, Suite 103, Columbia, SC 29210. E-mail: sbaxter@gwm.sc.edu. This research was supported by R01 grant HL 63189 from the National Heart, Lung, and Blood Institute of the National Institutes of Health. Suzanne Domel Baxter, PhD, RD, FADA was the principal investigator. The authors express appreciation to the children, faculty, and staff of Blythe, Goshen, Gracewood, Hephzibah, Lake Forest Hills, McBean, Monte Sano, National Hills, Rollins, Southside, Willis Foreman, and Windsor Spring Elementary Schools; to the School Nutrition Program; and to the Richmond County Board of Education in Georgia for allowing data to be collected. Direct observation of meals is often considered the "gold standard" by which dietary assessment tools are validated (1-3) because it is practical (4), economical (3), independent of the subject's memory (5,6), and can provide unbiased information about the subject's actual intake (7). For direct observation of meals, observers typically watch subjects throughout a defined period (eg, school lunch) and take notes on the subjects' eating behaviors regarding items and amounts consumed, traded (received/given away), and/or spilled. Examples of 26 studies in which direct observation has been used include studies to validate dietary recalls (5,6,(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22), food frequency questionnaires (11,23), and food/diet records (11,24) as well as studies to assess dietary intake of individuals (25-31) and to evaluate nutrition education interventions (31).
NIH Public AccessThe use of direct observation as a validation tool is based on the assumption that what ...