INTRODUCTION
The Centers for Disease Control and Prevention and the US Department of Agriculture, Food and Nutrition Service (USDA-FNS) endorse the use of policy, systems, and environmental (PSE) change interventions as public health approaches to addressing chronic diseases, including those resulting from poor diets and obesity.
1- Honeycutt S
- Leeman J
- McCarthy WJ
- et al.
Evaluating policy, systems, and environmental change interventions: lessons learned from CDC's Prevention Research Centers.
, In fact, USDA-FNS
Supplemental Nutrition Assistance Program Education (SNAP-Ed) Plan Guidance requires states to implement PSEs in addition to traditional direct nutrition education interventions that involve face-to-face contact with persons eligible for SNAP-Ed, those living in households with incomes ≤185% of the federal poverty level.
Policy, systems, and environmental interventions offer a more comprehensive approach to public health interventions than traditional direct education by recognizing and addressing influences external to the individual that can affect risk and prevention behaviors.
3- Savoie-Roskos MR
- DeWitt K
- Coombs C
Changes in nutrition education: a policy, systems, and environmental approach.
To date, most evaluations of PSE interventions have been descriptive and have relied on documenting lessons learned,
1- Honeycutt S
- Leeman J
- McCarthy WJ
- et al.
Evaluating policy, systems, and environmental change interventions: lessons learned from CDC's Prevention Research Centers.
,4- Nitta M
- Navasca D
- Tareg A
- Palafox NA
Cancer risk reduction in the US Affiliated Pacific Islands: utilizing a novel policy, systems, and environmental (PSE) approach.
, 5- Rohan EA
- Chovnick G
- Rose J
- Townsend JS
- Young M
- Moore AR
Prioritizing population approaches in cancer prevention and control: results of a case study evaluation of policy, systems, and environmental change.
, 6- Kegler MC
- Honeycutt S
- Davis M
- et al.
Policy, systems, and environmental change in the Mississippi Delta: considerations for evaluation design.
, 7- Hardison-Moody A
- Fuller S
- Jones L
- et al.
Evaluation of a policy, systems, and environmental-focused faith-based health promotion program.
, 8- Parsons AA
- Monteban M
- Lee E
- et al.
Indicators of readiness and capacity for implementation of healthy eating strategies in child care settings serving low-income children.
, 9- Fagen MC
- Asada Y
- Welch S
- et al.
Policy, systems, and environmentally oriented school-based obesity prevention: opportunities and challenges.
or have used in part or whole the reach, adoption, implementation, and maintenance framework.
10- Patel S
- Kwon S
- Arista P
- et al.
Using evidence-based policy, systems, and environmental strategies to increase access to healthy food and opportunities for physical activity among Asian Americans, Native Hawaiians, and Pacific Islanders.
, 11- Naja-Riese A
- Keller KJM
- Bruno P
- et al.
The SNAP-Ed Evaluation Framework: demonstrating the impact of a national framework for obesity prevention in low-income populations.
, 12- Bunnell R
- O'Neil D
- Soler R
- et al.
Fifty communities putting prevention to work: accelerating chronic disease prevention through policy, systems and environmental change.
Few studies have collected outcome data to assess the potential effectiveness of PSE interventions. Moreover, those studies that have connected PSEs to behaviors have not provided compelling evidence associating these interventions with improved outcomes. For example, Lepe and colleagues
13- Lepe S
- Goodwin J
- Mulligan KT
- Balestracci K
- Sebelia L
- Greene G
Process evaluation of a policy, systems, and environmental change intervention in an urban school district.
compared fifth-graders receiving PSE lessons with those receiving only classroom-based nutrition education, in addition to documenting the fidelity, reach, and acceptance of a school-based PSE intervention. The authors reported that students exposed to the PSE lessons had higher fruit and vegetable knowledge scores, but no difference was found when comparing children's behavior, which was measured as the number of times they ate fruit and vegetables during the previous day.
13- Lepe S
- Goodwin J
- Mulligan KT
- Balestracci K
- Sebelia L
- Greene G
Process evaluation of a policy, systems, and environmental change intervention in an urban school district.
One challenge to assessing outcomes regarding PSE change interventions outside of controlled settings such as a school includes accounting for non-PSE factors that may explain behavior change. Another challenge is measuring levels of PSE exposure, especially when individuals may not be aware that they were exposed to a PSE.
One approach to evaluating population-based interventions where levels of exposure cannot be accurately ascertained from members of the target population is to employ an ecological study design. In 1 such study, 2,907 census tracts with the number of Californians receiving SNAP-Ed direct education were linked to geocoded, population-based survey responses from SNAP-Ed eligible mothers.
14- Molitor F
- Sugerman SB
- Sciortino S
Fruit and vegetable, fat, and sugar-sweetened beverage intake among low-income mothers living in neighborhoods with Supplemental Nutrition Assistance Program-Education.
This study found that higher reach of direct education was significantly related to greater intake of fruits and vegetables and decreased consumption of sugar-sweetened beverages (SSBs).
14- Molitor F
- Sugerman SB
- Sciortino S
Fruit and vegetable, fat, and sugar-sweetened beverage intake among low-income mothers living in neighborhoods with Supplemental Nutrition Assistance Program-Education.
In the intervening years, the number of Californians participating in SNAP-Ed direct education interventions has decreased by 32.6%, whereas sites implementing PSE change interventions increased from 682 in 2015 to 1,566 in 2018 in California.
These changes have been driven by the USDA
directive, as well as the cost savings of PSE changes over direct education interventions during a time of declining SNAP-Ed funding.
The California Family Health Study is an annual statewide survey designed to track dietary behaviors and diet quality among persons from SNAP-Ed eligible households. The current study linked interview responses from adult caregivers participating in the 2018 California Family Health Study to the number of PSE change interventions implemented in their neighborhoods to examine dietary behaviors and diet quality regarding potential exposure to PSE change interventions. The goal of the current ecological study was to determine whether the empirical evidence supports the continued investment in PSE change interventions by the Centers for Disease Control and Prevention, the UDSA-FNS, and the California CalFresh Healthy Living (SNAP-Ed) program.
METHODS
The current study was reviewed and approved by the California Health and Human Services Agency, Committee for the Protection of Human Subjects. Oral consent was documented for all survey participants.
Households with incomes estimated to be ≤185% of the federal poverty level, with at least 1 child aged 5–17 years, were sampled at random from within all 58 California counties using the California Department of Health Care Services, Medi-Cal Eligibility Data System. Recruitment and screening procedures included an initial letter in English or Spanish, followed by a phone call to identify the caregiver. Specifically, bilingual interviewers screened for the youngest adult in the household who “assumes the responsibility in caring for the health and well-being of the child(ren) and prepares the meals or buys food for the child(ren).”
Before the dietary interviews, households were sent pictorial food and beverage portion-size booklets and measuring cups and spoons. The quantity and size of each food and drink item consumed for each meal and snack over the prior 24 hours were assessed using the Automated Self-administered 24-Hour Dietary Assessment Tool (ASA24),
while trained interviewers referred caregivers to portion-size pictures or measuring cups and spoons. Portion sizes for the reported items were entered into the Web-based ASA24 system. Responses to standardized demographic items were recorded separately. Survey participants received a $15 (USD) gift card.
Five dietary outcome variables, in line with California's SNAP-Ed statewide program goals and objectives,
were examined:
Fruits and vegetables (cups) were whole or cut fruits and dark green, red, orange, or starchy vegetables.
Water (cups) included tap or unsweetened bottled water, including flavored or vitamin-fortified water. The
SSBs (cups) represented sugar-sweetened soda, energy, fruit, sports, coffee, or tea beverages.
Added sugars (teaspoons) were sugar (white, brown, and raw), syrup (corn, high fructose, malt, maple, and pancake) and fructose sweetener, liquid fructose, honey, molasses, dextrose, and dextrin that were eaten separately or as ingredients from processed or prepared foods. Diet quality was assessed by the
Healthy Eating Index (HEI)
–2015 composite scores.
Healthy Eating Index scores range from 0 to 100, with higher values aligning with greater adherence to key recommendations of the 2015–2020 Dietary Guidelines for Americans
. These 5 outcome variables are not mutually exclusive. Fruits, vegetables, and added sugars are included in the 13 component scores used to calculate total HEI–2015 scores.
In 2018, all
CalFresh Healthy Living program contractors across California entered information on PSE change and direct education interventions into the Program Evaluation and Reporting System.
Using these data, the physical addresses of all PSEs and direct education activities were geocoded to the census tract level. A file was then created identifying, by census tract, PSE and direct education reach, that is, the number of PSE change interventions and the number of direct education participants within each census tract. Direct education was included in this file to serve as a control variable because the reach of direct education activities has been associated with more healthful SSB and fruit and vegetable intake.
14- Molitor F
- Sugerman SB
- Sciortino S
Fruit and vegetable, fat, and sugar-sweetened beverage intake among low-income mothers living in neighborhoods with Supplemental Nutrition Assistance Program-Education.
These data were merged by census tract numbers with the survey response file based on the geocoded addresses where the 2018 California Family Health Study participants lived. The final analytic dataset excluded caregivers living in census tracts without PSE change and/or direct education interventions. This approach facilitated focusing the analyses on caregivers living in geographic areas where SNAP-Ed interventions had occurred, rather than on the larger sample of 2018 California Family Health Study participants representing SNAP-Ed eligible caregivers living in census tracts with and without SNAP-Ed interventions.
The independent variable for the current study was coded as 0, 1, or ≥2 PSE change interventions per census tract. Initially, ANOVA was used to compare mean values for the 5 outcome variables across the 3 PSE groups, and Cohen d was calculated to determine the effect size for ≥2 vs 0 PSE change interventions. Regression analyses were then performed to control for caregivers’ age and race and/or ethnicity, and the reach of direct education within the census tracts. Descriptive analyses found cups of fruits and vegetables, water, and SSBs, and teaspoons of added sugars to be highly skewed to the right (skewness >2). Accordingly, these variables were examined by the ordinal PSE variable in both linear and log-linear regression analyses. HEI–2015 scores (skewness = −0.80) were examined using linear regression only. The covariates represented the number of direct education participants in each census tract, caregivers’ age (centered on the mean), and race and/or ethnicity: Latino (“Hispanic, Latina, or of Spanish origin”) was the reference group for White people, African Americans, and Other and/or Missing (responses to other ethnic categories or missing data).
Excluded from all analyses were 74 records representing partial ASA24 interviews. Differences were considered significant at the 0.05 alpha level. The criteria of 0.2, 0.5, and 0.8 were used for interpreting Cohen d as small, medium, and large effects, respectively. Data merging, cleaning, coding, and analyses were conducted with SPSS (version 26.0, IBM Corp, Armonk, NY, 2019).
RESULTS
A total of 2,222 caregivers provided complete ASA24 interviews and lived in a census tract with direct education and/or PSE change interventions. One fifth (21.2%) of caregivers were male; 69.1% of the sample was Latino (
Table 1). The average age of survey participants was 38.1 years, and 16.8% had less than an eighth-grade education.
Table 1Characteristics of Caregivers from SNAP-Ed Eligible Households Across California, 2018 (n = 2,222)
SNAP-Ed indicates Supplemental Nutrition Assistance Program Education.
The 2,222 caregivers lived in 1,123 unique census tracts. A total of 170 caregivers lived in census tracts with no SNAP-Ed direct education activities; an average of 272 persons per census tract across the 1,123 census tracts received direct education. In addition, the 2018 Program Evaluation and Reporting System reported 500 PSE change interventions in the 1,123 census tracts, representing 30.0% of the 1,665 PSEs occurring in California in 2018. The most common, overlapping focus areas of the 500 PSE change interventions were fruits and vegetables (66.4%), the USDA MyPlate
food groups and portion sizes (36.0%), intake of water (24.8%), and limiting added sugars (14.4%;
Table 2). Six in 10 (57.6%) caregivers lived in a census tract with no PSEs, 28.7% lived near 1, and 13.7% lived near 2 or more PSE change interventions.
Table 2Focus Areas of PSE Change Interventions (n = 500) Implemented in Census Tracts of Caregivers Participating in the 2018 California Family Health Study (n = 2,222)
PSE indicates policy, systems, and environmental; USDA, US Department of Agriculture.
Note: Focus Areas are reported as recorded in the Program Evaluation and Reporting System and are not mutually exclusive.
The number of PSE change interventions per census tract was related to a decreased intake of SSBs (Cohen
d = −0.12) and added sugars (Cohen
d = −0.18), and increased HEI–2015 scores (Cohen
d = 0.18), regardless of caregivers’ race and/or ethnicity or age, or the reach of direct education in one's neighborhood (
Table 3). Cups of fruits and vegetables and water were not related to the reach of PSE change interventions. As seen in
Table 3, these findings were consistent between the linear and log-linear regression models, suggesting that the non-normal distribution of the 4 dietary behavior measures did not influence the identification of significant correlates of PSE reach. On average, SNAP-Ed eligible caregivers living in census tracts with ≥2 PSE interventions consumed 0.2 fewer cups (1.6 ounces) of SSBs and 2.2 fewer teaspoons of added sugars per day and had a 2.6 higher HEI–2015 score, than caregivers without PSE interventions in the census tracts where they lived. Finally, the correlation between the number of PSE change interventions and direct education participants was significant (
P ≤0.001), but weak (
r = 0.16,
R2 = 0.025), indicating that multicollinearity did not overly influence the regression models.
Table 3Caregivers’ Dietary Behaviors and HEI–2015 Scores by Number of PSE Sites per Census Tract, 2018
CI indicates confidence interval; HEI, Healthy Eating Index; PSE, policy, systems, and environmental; SSB, sugar-sweetened beverage.
DISCUSSION
The PSE change interventions implemented in the census tracts of a random sample of caregivers across California in 2018 focused mostly on fruits and vegetables, the USDA MyPlate
recommendations, drinking water, and limiting added sugar. These interventions appear to have been more effective in reducing the intake of added sugars overall and SSBs in particular than increasing the consumption of fruits and vegetables or water. Perhaps PSEs intended to address added sugars were better designed or were directed at populations in greater need of reducing sugar intake than those aimed at SNAP-Ed eligible individuals who believed that they were already consuming sufficient quantities of fruits, vegetables, and water. Alternatively, PSEs focusing on limiting the consumption of sugary foods and beverages may have resonated more with the SNAP-Ed eligible caregivers for reasons that included a greater understanding of the link between sugar, rather than fruits and vegetables or water, and the risk of obesity and the related health consequences for the children under their care. Moreover, PSE efforts to increase the intake of fruits and vegetables may be circumvented by the limited availability of fresh produce at home among low-income populations.
22Association of individual and neighborhood factors with home food availability: evidence from the National Health and Nutrition Examination Survey.
Another explanation is that caregivers may not have responded to PSEs because they were already saturated with messages about eating more fruits and vegetables.
Of greater importance is the finding that caregivers in neighborhoods with PSE interventions had higher HEI–2015 scores, and thus were more likely to report diets in adherence with the 2015–2020 Dietary Guidelines for Americans
. This finding may be because over one-third of PSE change interventions focused on the USDA MyPlate
recommendations, which are based on the Dietary Guidelines for Americans
. Prior research has linked a lower risk of the incidence of and death from cardiovascular disease and a lower risk of all-cause mortality with increased HEI–2015 scores.
23- Hu EA
- Steffen LM
- Coresh J
- Appel LJ
- Rebholz CM
Adherence to the Healthy Eating Index-2015 and other dietary patterns may reduce risk of cardiovascular disease, cardiovascular mortality, and all-cause mortality.
Given that these protective effects may be benefiting caregivers from higher-reach PSE neighborhoods, and the generalizability of the results from a statewide, population-based survey, the current study offers the strongest evidence to date in support of the potential beneficial impact of PSEs.
The USDA-FNS categorizes SNAP-Ed approaches as individual or group-based direct education, multilevel interventions, and community and public health interventions.
The USDA-FNS mandates that SNAP-Ed include a combination of these approaches, per the Social Ecological Model, which posits that individual behavior change and resulting health outcomes are best achieved by addressing (through programs or policies) different influential levels of society, such as communities and organizations.
, Specifically, the USDA-FNS requires states to implement 2 of 3 approaches, with PSE change interventions, as multilevel or public health approaches, accompanying direct education.
The significant correlations presented in the current study between residing near 1 or more PSEs and decreased consumption of SSBs and added sugars, and improved overall diet quality, regardless of levels of direct education, suggest that individual or group-based education interventions may not be required to change nutrition behaviors. Thus, multilevel interventions could exclude direct education, and community and public health interventions without direct education may still be effective in changing unhealthful dietary behaviors.
The temporal sequence between the independent variable and outcome measures is unknown and represents the most noteworthy limitation of the current study. In addition, covariates other than those included in the statistical models, such as local or regional non-SNAP-Ed efforts to improve dietary behaviors among low-income populations, might explain the findings. Shortcomings of the study also include self-reported information on dietary intake and not being able to identify which PSE interventions were more effective than others, and in turn identify those that meet the criteria for
research-based PSE interventions.
Finally, with Cohen
d for SSBs, added sugars, and HEI–2015 scores failing to meet the criterion for small effects, the implications of the findings must be interpreted with caution.
Article info
Publication history
Published online: July 14, 2020
Accepted:
May 28,
2020
Footnotes
Conflict of Interest Disclosure: The authors have not stated any conflicts of interest.
Copyright
© 2020 Society for Nutrition Education and Behavior. Published by Elsevier Inc.