I read with great interest the JNEB article titled “A Multicomponent, School-Based Intervention, the Shaping Healthy Choices Program, Improves Nutrition-Related Outcomes,” by Scherr and colleagues.
1
The results reported by the authors with regard to body mass index (BMI) reductions are dramatic and exciting.Unfortunately, in my estimation, they are also likely wrong. In a persuasive letter to the editor,
2
Wood and colleagues made a case of how the reported analyses are inappropriate for the study's cluster randomized trial design. As I see it, the issue is not simply one of minor methodological limitation. The issue is one of invalidation.Cluster randomized trials that do not account for their clustering risk type I, or false-positive, error.
3
, 4
Such error results from the fact that individuals in a given cluster may share characteristics with each other (eg, social, cultural, behavioral), making them more similar to each other than to individuals in other clusters. Individuals in the study by Scherr et alNational Institutes of Health. Research Methods Resources: Group- or cluster-randomized trials (GRTs). https://researchmethodsresources.nih.gov/grt.aspx. Accessed April 12, 2018.
1
were fourth-graders and clusters were California schools. Given the small number of clusters (n = 4 schools), the ability to estimate within-cluster similarity is limited,4
but that does not mean that within-cluster similarity can be ignored. The authors mentioned calculating an intraclass (within-cluster) correlation coefficient, but based on the information presented in the paper, then did not take such correlation into account in their main analyses: the ones showing the dramatic decreases in BMI percentiles. Instead the authors used individual-level analyses. So the result is that their findings of reductions in BMI percentile may have been inflated, or potentially, just plain false.National Institutes of Health. Research Methods Resources: Group- or cluster-randomized trials (GRTs). https://researchmethodsresources.nih.gov/grt.aspx. Accessed April 12, 2018.
4
, National Institutes of Health. Research Methods Resources: Group- or cluster-randomized trials (GRTs). https://researchmethodsresources.nih.gov/grt.aspx. Accessed April 12, 2018.
5
The letter to the editor by Wood et al
2
makes similar arguments. Based on the reply from Scherr et al,6
it is my impression that they seem to misunderstand the issue completely. The problem is not with regard to sample size, per the authors’ defense; it is related to nesting, intraclass (within-cluster) correlation, and valid analyses.Fortunately, the analyses can be corrected, and indeed, I believe the Wood group
2
has both the expertise and the willingness to assist the authors in this regard.The bottom line is that the currently published work cannot stand uncorrected as part of the extant literature.
7
I believe the published article may very well be invalid and it must be corrected or retracted.Acknowledgments
The author is supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health under Award K23HD079606. The content of this publication is solely the responsibility of the author and does not necessarily represent the official views of the National Institutes of Health.
References
- A multicomponent, school-based intervention, the Shaping Healthy Choices Program, Improves nutrition-related outcomes.J Nutr Educ Behav. 2017; 49 (e1): 368-379
- A comment on Scherr et al “A multicomponent, school-based intervention, the Shaping Healthy Choices Program, improves nutrition-related outcomes.”.J Nutr Educ Behav. 2018; 50: 324-325
- Comparison of tests for categorical data from a stratified cluster randomized trial.Stat Med. 2002; 21: 3835-3846
National Institutes of Health. Research Methods Resources: Group- or cluster-randomized trials (GRTs). https://researchmethodsresources.nih.gov/grt.aspx. Accessed April 12, 2018.
- Developing appropriate methods for cost-effectiveness analysis of cluster randomized trials.Med Decis Making. 2012; 32: 350-361
- Response to “A comment on Scherr et al ‘A multicomponent, school-based intervention, the Shaping Healthy Choices Program, improves nutrition-related outcomes.”.J Nutr Educ Behav. 2018; 50: 326-327
- Design and analysis of group-randomized trials: a review of recent practices.Am J Public Health. 2004; 94: 393-399
Article info
Publication history
Published online: August 01, 2018
Accepted:
June 28,
2018
Received in revised form:
April 13,
2018
Received:
March 29,
2018
Footnotes
Conflict of Interest Disclosure: The authors have not stated any conflicts of interest.
Identification
Copyright
© 2018 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.
