Abstract
Objective
To evaluate inter-coder (between-coder) and intra-coder (within-coder) reliability
among trained data coders who enter 24-hour dietary recall data collected through
Expanded Food and Nutrition Education Program operations in the state of Georgia.
Design
This study employed multiple cross-sectional evaluations of inter-coder reliability
and a short-term longitudinal evaluation of intra-coder reliability.
Participants/Setting
Study participants consisted of trained data coders (n = 9) who were employed during
the 12-month period of evaluation.
Main Outcome Measures
Primary outcome measures were inter-coder and intra-coder reliability across data
entered into the Web-based Nutrition Education Evaluation and Reporting System. Statistical
analyses were conducted using IBM SPSS 24. Descriptive statistics were generated and
inter-coder and intra-coder reliability were assessed using 2-way mixed intraclass
correlation coefficients.
Results
Results of this evaluation indicated good to excellent inter-coder reliability among
all coders, and excellent intra-coder reliability among the majority of coders. However,
some notable inconsistencies were identified within the intra-coder reliability analyses.
Conclusions
Future strategies to improve data quality within Expanded Food and Nutrition Education Program operations might include enhanced training for data coders, implementation of error
detection protocols, expansion of the Web-based Nutrition Education Evaluation and
Reporting System database, and exploration of automated, computer-assisted administration
of 24-hour dietary recalls.
Key Words
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessSNEB Member Login
SNEB Members, full access to the journal is a member benefit. Login via the SNEB Website to access all journal content and features.Subscribe:
Subscribe to Journal of Nutrition Education and BehaviorAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- 20years of EFNEP: changes and challenges.J Nutr Educ Behav. 1990; 21: 265-269
- Collection methods for the 24-hour dietary recall as used in the Expanded Food and Nutrition Education Program.J Nutr Educ Behav. 2017; 49 (250–256.e251)
US Department of Agriculture, National Institute of Food and Agriculture. https://nifa.usda.ogv/tool/webneers. Accessed January 4, 2018.
- Update on NHANES dietary data: focus on collection, release, analytical considerations, and uses to inform public policy.Adv Nutr. 2016; 7: 121-134
- Improving the quality of data from EFNEP participants with low literacy skills: a participant-driven model.J Nutr Educ Behav. 2014; 46: 309-314
- Preventing human error: the impact of data entry methods on data accuracy and statistical results.Comput Hum Behav. 2011; 27: 1834-1839
US Department of Health and Human Services, US Department of Agriculture. 2015–2020 Dietary Guidelines for Americans. 8th edition. December 2015. https://health.gov/dietaryguidelines/2015/guidelines/. Accessed January 31, 2019
- Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology.Psychol Assess. 1994; 6: 284-290
- Methods of dietary and nutritional assessment and intervention and other methods in the Multiple Risk Factor Intervention Trial.Am J Clin Nutr. 1997; 65: 196S-210S
- Computer-assisted self-interviewing: a multimedia approach to dietary assessment.Am J Clin Nutr. 1997; 65: 1275s-1281s
- The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute.J Acad Nutr Diet. 2012; 112: 1134-1137
- Comparison of interviewer-administered and automated self-administered 24-hour dietary recalls in 3 diverse integrated health systems.Am J Epidemiol. 2015; 181: 970-978
- Performance of the Automated Self-Administered 24-hour Recall relative to a measure of true intakes and to an interviewer-administered 24-h recall.Am J Clin Nutr. 2014; 100: 233-240
- Nutritional Epidemiology.2nd ed. Oxford University Press, New York, NY1998
- Disparities in state-specific adult fruit and vegetable consumption—United States, 2015.MMWR Morb Mortal Wkly Rep. 2017; 66: 1241-1247
- Comparison of 4 nutrient databases with chemical composition data from the Dietary Approaches to Stop Hypertension trial. DASH Collaborative Research Group.J Am Diet Assoc. 1999; 99: S45-S53
- Interrater agreement and interrater reliability: key concepts, approaches, and applications.Res Social Adm Pharm. 2013; 9: 330-338
- Reliability, repeatability and reproducibility: analysis of measurement errors in continuous variables.Ultrasound Obstet Gynecol. 2008; 31: 466-475
- Improving data quality control in quality improvement projects.Int J Qual Health Care. 2009; 21: 145-150
Article info
Publication history
Published online: February 06, 2019
Footnotes
Conflict of Interest Disclosure: The authors have not stated any conflicts of interest.
Identification
Copyright
© 2019 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.
