Abstract
Objective
To explore California local health department leaders’ experiences planning, implementing, and evaluating nutrition promotion and obesity prevention programs for low-income families.
Design
Qualitative, cross-sectional study using semi-structured in-depth interviews and panel interviews conducted in 2015–2016.
Setting
California local health departments (LHDs) funded by the California Department of Public Health to implement Supplemental Nutrition Assistance Program–Education (SNAP-Ed).
Participants
The authors recruited SNAP-Ed leaders from all 58 California LHDs implementing SNAP-Ed. Leaders from 49 LHDs participated: 36 in hour-long, in-depth interviews and 13 in 1 of 3 90-minute group panel interviews.
Phenomenon of Interest
Processes, facilitators, and barriers connected to delivering SNAP-Ed reported by leaders in planning, implementing, and evaluating local programs.
Analysis
Interviews were transcribed, coded, and analyzed using Dedoose software.
Results
Leaders grappled with introducing, implementing, and integrating policy, systems, and environmental change interventions (PSEs). Information used to make planning decisions varied widely across LHDs. Partnership with nontraditional organizations was described as a resource- intensive, nonlinear process with recognized potential for benefit. Rural programs reported specific and different experiences compared with their urban counterparts.
Conclusions and Implications
Implementing new, complex interventions to improve diet and activity environments and behaviors is both exciting and challenging for local leaders. They expressed a desire for additional resources and capacity building to facilitate success, particularly related to policy, systems, and environmental change programs. Attention to the specific needs of rural counties is needed.
Key Words
Introduction
Responding to the obesity epidemic in the US, researchers, practitioners, governments, and the private sector have emphasized the importance of changing physical and social environments to facilitate healthy eating and active living.
1
, 2
, 3
Although interventions and promising strategies were documented in the literature,4
insufficient scientific study addressed the delivery of public health programs, leaving gaps in understanding practitioners’ practices and needs, and the best ways to optimize and improve nutrition program delivery.5
, 6
, 7
, 8
Menon et al6
asserted that optimizing the impact of evidence-based interventions requires more research on program implementation and operational strategies, particularly planning, decision-making, management, training methods, and workforce capacity.The Supplemental Nutrition Assistance Program–Education (SNAP-Ed), which is funded by the US Department of Agriculture's Food and Nutrition Service,
9
provides the largest annual nationwide investment to states to support nutrition education and obesity prevention in low-income communities. In California, the Department of Social Services distributes SNAP-Ed funds to 4 agencies; the majority of funding goes to the Department of Public Health (CDPH). In 2013, CDPH shifted the local funding strategy to provide SNAP-Ed funds to local health departments (LHDs) rather than directly to community organizations, as it had historically. Wu et alUS Department of Agriculture Economic Research Service. Funding allocations. FY 2018 final state allocations. https://snaped.fns.usda.gov/program-administration/funding-allocations. Accessed December 17, 2018.
10
evaluated this new funding model and found that many activities such as strategic planning, partnerships, and subcontracting had moved closer to the targeted communities. At the same time, the study found that local administrative burden was high and health departments had limited ability to tailor programs owing to state and federal constraints.In tandem with the funding model shift, SNAP-Ed program priorities and guidance evolved after a change in the Healthy, Hunger-Free Kids Act of 2010 requiring SNAP-Ed programs to emphasize obesity prevention in addition to nutrition education. This established a new focus on implementing policy, systems, and environmental change interventions (PSEs), such as efforts to improve access to healthy foods in low-income neighborhoods or improve school food environments.
11
Local health departments receiving SNAP-Ed funds in California were universally working to integrate nutrition education with environmental and policy change initiatives.US Department of Agriculture Supplemental Nutrition Assistance Program–Education (SNAP-Ed) factsheet. https://snaped.fns.usda.gov/snap/SNAP-Ed%20Factsheet%20_August%202016.pdf. Accessed March 22, 2019.
This study examined the processes, successes, and challenges of planning, implementing, and evaluating these SNAP-Ed programs. It aimed to fill a gap by illuminating the processes of delivering nutrition education and obesity prevention programs as described by LHD leaders tasked with delivering these programs in California. The study provides insights for practitioners, funders, and researchers supporting, delivering, and studying these complex programs.
Methods
This cross-sectional qualitative study used a 2-stage data collection process to explore California LHD leaders’ experiences delivering SNAP-Ed programs. In stage 1, the researchers conducted semi-structured interviews with LHD leaders. In stage 2, 3 panel (group) interviews were conducted by phone with LHD leaders who were not initially interviewed.
Setting
The study was conducted from November, 2015 to May, 2016 with 49 of 58 SNAP-Ed leaders from LHDs in California. At that time, LHDs were early in the process of integrating PSEs with long-standing nutrition education programs. They represented the diversity of California's SNAP-Ed–funded LHDs by geography (categorized as urban, small to midsize urban, and rural) and program size (using budget as a proxy).
The University of California, Davis, Institutional Review Board determined that this project was not human subjects research.
Conceptual Framework
A nutrition education framework, which connects theory and practice,
12
was adapted to fit the SNAP-Ed program's multiple components, outlining connections between expected inputs and outputs (Supplementary Data). This framework served as an initial guide to instrument development and data analysis. It evolved with the researchers’ understanding of the intervention and its processes.Instruments
The study team developed a semistructured interview guide that explored program leaders’ experiences delivering SNAP-Ed (Table 1), informed by the conceptual framework and the unpublished findings of 2 previously conducted evaluations.
12
The researchers explored planning; decision-making; collaboration; challenges and successes; program evaluation; and staff capacity, administrative, and training and technical support needs. The interview guide was pilot-tested with a recently retired SNAP-Ed local leader and was also refined after early interviews. The panel interview guide was developed after interviews were conducted to test and enrich preliminary interview findings.Table 1Example Questions from Semi-Structured Interview Guide
| Planning |
| 1. Thinking about planning your Supplementation Nutrition Assistance Program–Education (SNAP-Ed) work, how do you decide what to focus on in your work plan? |
| 2. We are also interested in learning more about how collaboration occurs as part of SNAP-ED planning. In developing your integrated work plans, how, if at all, have you collaborated with local implementing agencies, sub-contractors, and County Nutrition Action Plan (CNAP) to develop your plan? |
| 3. Would you describe approaches that worked particularly well in collaborative planning with any or all of the groups we have discussed? |
| 4. What, if any, difficulties has your local health department (LHD) had in collaborating to develop the Integrated Work Plan (IWP)? |
| Implementation |
| 5. Could you start by describing which SNAP-Ed activities your local health department (LHD) is working on currently that you are most excited about? |
| 6. Some SNAP-Ed activities are more difficult to implement than others. Which SNAP-Ed activities have been most challenging for your LHD to implement and why? |
| 7. We are particularly interested in learning more about Policy, Systems and Environmental change (PSE) work in your LHD. In doing your PSE work, do you have an example of a good partnership your LHD has formed and why it has worked well? |
| Evaluation |
| 8. What training or support on evaluation does your LHD need? |
| Support |
| 9. What skills and expertise would you most like to develop in your SNAP-Ed team in the near future? |
Participant Recruitment
The study team recruited local SNAP-Ed project directors for study participation. These were individuals primarily responsible for the delivery and administration of SNAP-Ed in their counties. In some cases, project directors included additional staff in interviews; thus, study participants are referred to here as local leaders, to include both project directors and the key staff they included. Maximum variation sampling and convenience sampling were used to ensure the participating LHDs included diverse budget levels across varied geographic settings (urban, rural, and midsize). The 58 LHDs with SNAP-Ed funding were divided into 2 groups, each composed of roughly equal numbers of LHDs of 3 budget sizes (small = <$500,000/y, midsize = $500,001 to $1,000,000/y, and large = >$1 million/y) and 3 settings (categorized as large urban, small to midsize urban, or rural, per Centers for Disease Control and Prevention
13
definition). The research team invited project directors from the first group to sign up for interviews from November to December, 2015, and project directors from the second group from December, 2015 to January, 2016. In this way, whereas only LHDs that chose to participate were part of the sample (convenience), sample variation was ensured (maximum variation) before the point of saturation was reached (the point at which information being elicited was consistent with what had already been learned).Centers for Disease Control and Prevention. NCHS Urban-Rural Classification Scheme for Counties. National Center for Health Statistics. https://www.cdc.gov/nchs/data_access/urban_rural.htm#2013_Urban-Rural_Classification_Scheme_for_Counties. Accessed March 22, 2019.
The recruitment e-mail to LHD project directors included a study description and an interview sign-up link. The researchers followed up by phone with directors who had not yet agreed to participate. Those who signed up for interviews received automated reminders to facilitate interview completion.
After 36 interviews with relatively equal participation of LHDs from each category (rural, urban, and midsize), the researchers determined that no new information was being elicited and stopped that type of data collection. After preliminary analysis, SNAP-Ed project directors who had not yet been interviewed were invited to participate in panel (group) interviews to assess whether preliminary results were consistent with their experiences, in particular, whether differences observed by setting type (rural, urban, and midsize) were accurate. Panel interviews focused on asking structured questions about distinct experiences by setting type, owing to budgetary and geographic differences, with follow-up prompts as needed. The researchers conducted recruitment using e-mails and phone calls. Three panels (rural, urban, and midsize, respectively) were conducted using a secure conference line with groups of LHD leaders.
Data Collection
All members of the research team received training on data collection methods, including a review of the study protocol, objectives, and instruments, as well as interview practice and observation sessions. All had previous experience conducting interviews and qualitative research. Initial questions were read verbatim; probes were used as needed. All data were collected using a secure phone line, audiotaped, and professionally transcribed. Interviews lasted 1 hour; panel interviews took 90 minutes. Transcripts were imported into Dedoose (version 6.2.10), secure online qualitative software that facilitates coding (particularly for multiple coders) and review of excerpts and allows for memo writing.
Analysis
At the outset of analysis, all members of the research team reviewed the analytic process outlined in the study protocol and discussed the approach. A codebook was developed using a priori categorical codes and subcodes drawn from the study objectives, the nutrition education conceptual framework, and the interview guide. A priori codes represented primary areas of interest and key public health program functions (eg, planning, implementation, and evaluation). A priori subcodes included challenges, opportunities, and process. As analysis progressed, a set of emergent codes and subcodes that were not originally anticipated were developed, integrated into the codebook, and applied.
To ensure coding consistency and rigor, the research team opted for an approach that emphasized sustained efforts throughout the duration of the study to ensure rigor and coding consistency, using codebook refinement, ongoing assessments of intercoding reliability, and frequent discussion, an approach informed by Barbour's
14
exploration of rigor in qualitative research. This included multiple coders coding initial interview transcripts: 3 research team members (H.D.S., J.A., and J.O.) coded early transcripts and reviewed code applications. Once coders were reliably consistent, a single researcher coded each remaining transcript. Subsequently, the 2 primary coders met weekly to review 100-line excerpts randomly selected from transcripts to ensure coding remained consistent. Few coding differences arose, but when they did, differences were aligned during discussions and codebook definitions were made more specific. The full research team met weekly to discuss coding differences, synthesize codes, discuss themes, and present findings. Memos and triangulation between interview and panel interviews data completed analysis and served to test the validity of findings.Results
Of the 58 local health departments, 49 recruited participated (Table 2), 36 participated in interviews, and 13 participated in panels.
Table 2Local Health Department (LHD) Setting Type and Funding Level, by Data Collection Method
| Urban–Rural Classification and size of Supplemental Nutrition Assistance Program–Education Budget | All Funded LHDs | LHD Participation, by Data Collection Methods | ||
|---|---|---|---|---|
| Interview Sample | Panel Sample | Total Sample | ||
| Large urban | ||||
| >$1,000,000 | 11 | 7 | 4 | 11 |
| $500,001-$1,000,000 | 4 | 3 | 1 | 4 |
| <$500,000 | 4 | 3 | 0 | 3 |
| Total urban | 19 | 13 | 5 | 18 |
| Small to midsize urban | ||||
| >$1,000,000 | 7 | 2 | 3 | 5 |
| $500,001-$1,000,000 | 10 | 8 | 1 | 9 |
| <$500,000 | 4 | 2 | 1 | 3 |
| Total small/midsize | 21 | 12 | 5 | 17 |
| Rural | ||||
| >$1,000,000 | 0 | 0 | 0 | 0 |
| $500,001-$1,000,000 | 1 | 1 | 0 | 1 |
| <$500,000 | 17 | 10 | 3 | 13 |
| Total not urban | 18 | 11 | 3 | 14 |
| Totals | 58 | 36 (62%) | 13 (22%) | 49 (84%) |
Findings emerged in 4 main areas: (1) experiences with the introduction, integration and implementation of PSE; (2) processes that leaders used in planning, implementing, and evaluating programs; (3) ways in which context, particularly rural vs urban setting, influenced planning, implementation, and evaluation; and (4) types of support requested to deliver programs.
Policy, Systems, and Environmental Change Interventions
Nearly all interviewees discussed the introduction, integration, and implementation of PSE interventions, focusing on 1 major idea.
Excitement coupled with lack of capacity
When asked what part of SNAP-Ed was most exciting, interviewees consistently discussed PSE efforts. In the following excerpt, the interviewee compares the impact of PSE with direct education efforts:
When we're looking at prevention and the public's health, by implementing policies and systems in the environment, where we're creating the healthy choice as the easy choice, we're impacting the public's health much greater and in a more direct broad manner than providing that direct education to that one individual or 10 individuals.
Although local leaders expressed excitement about this broader focus, several described a lack of knowledge or capacity to implement such efforts or alluded to the time required to make this change. Referring to the introduction of PSE into SNAP-Ed programming, which was historically focused on direct education, 1 leader said:
I think it's taking us some time to get up and running and really understand what that work [PSE] means and trying to figure out how to implement it within our organization and our county. … It's been more challenging for us than doing nutrition education.
Others alluded to a lack of clarity about which approach was a priority:
As we go into PSE work, there are going to be some people who are concerned, like, are we moving away from direct nutrition education, which was what SNAP-Ed was designed initially for …
Others highlighted practical realities of integrating policy, systems, and environmental change interventions alongside direct education efforts:
Though our staff is doing a lot of, like, nutrition education, it's a bit of a challenge to incorporate PSE work and nutrition education simultaneously, even though they go hand in hand.
Another said,
We need to work on maybe not being so segregated as, ‘I'm the dietician and this person does the PSE work,’ which is something we're working toward …
Another leader connected this idea of compartmentalization with how evaluation was supposed to be conducted across LHDs, referring to standardized evaluation protocols and frameworks:
Right now, we evaluate our PSE work through [Reach Effectiveness Adoption Implementation Maintenance] and nutrition education work through Impact Outcome Evaluation, and I think that … there's probably a better way … like looking at that more comprehensively and saying, ‘Okay, what are some indicators of success?’
In addition to addressing the challenge of PSE head on, local leaders frequently alluded to grappling with how to plan, implement, and evaluate PSE, the newer, broader approach to obesity prevention, as discussed next.
Processes
Asked about planning, implementation, and evaluation, local leaders described on-the-ground realities of program delivery: efforts that were variable, complex, and sometimes disjointed.
Marked variability in planning processes
The level of variation described in decision-making was particularly notable. Leaders reported weighing many types of information, including community needs, partnering agency resources, feasibility, policy windows, local politics, community acceptability, and funding levels. One interviewee described the decision-making process for developing work plans within the program:
How we're doing it is utilizing staff input … input from the community … our partnerships with other SNAP-Ed–funded and other partnerships in the community … thinking about where the gaps … are covered, what can we serve, what can they serve … and really thinking about how our internal LHD—we have other funding, not just SNAP-Ed funding—how our other nutrition programs can collaborate and build upon each other's work.
Another described more limited inputs:
I think what we're excited about is that we tweaked our work plan based on our previous year's work … we are trying to focus more. At first, before maybe it was the numbers [referring to the number of individuals reached], now we're settling down and say, “You know what? … Yes, we get all these numbers, but which ones are the ones that … we can have the best bang for our buck?
Although leaders considered many important sources of information for making decisions, few mentioned consulting the scientific literature.
Time and labor-intensive partnerships
Leaders shared examples of their investments in partnerships, some of which led to successful and innovative intervention efforts. They characterized nontraditional partnerships (eg, stores or retail environments, worksites) needed for implementing PSEs as a multi-step process that involved making contact, establishing trust, garnering buy-in, and aligning interests. For many, this was laborious and time-intensive, nonlinear, and complicated by multiple setbacks, including frequent administrative changes, lack of interest, and competing with other priorities at the partnering organization. This leader highlighted the time-intensive nature of developing a partnership with a rural school:
We're talking about a very small, pretty rural area where that just presents in itself challenges because you're a new outsider and people don't know you, trust you, know what your agenda is. So, it really took a year of spending time up there getting to know people and people getting to know me to understand what I was all about and what I could do to help them. … There's some good traction after really that year of having to kind of gain that trust to be there. Now, there's excitement and there are changes happening. It's pretty cool.
Another highlighted issues in establishing a relationship with a disinterested partner:
… trying to get into some of the bigger box stores like [X], for example … but they're so corporate that it's been really hard to establish a relationship there.
Others discussed the complexity of frequent staff changes, local politics, and maintaining momentum:
The [school] district we found has been pretty difficult to work with because of just the constant administrative changes and the district politics that come into play.
Another, discussing partnership with a business, said:
Some of those challenges … have to do with the fact that it's very labor-intensive to work with worksites, to try to get them on board with their worksite wellness efforts. Many times, we've been successful, and we have some momentum. Then, if there's some change at the organization, whether it's our contact person or maybe they have a new CEO or some change, then it sort of stalls that momentum that we had. It could be years’ worth of work that we've had going on, and then all of a sudden, it comes to a halt.
A number of local leaders discussed the particular challenge of aligning interests with retail or corner stores that sold produce and other goods, or grocery stores:
We're just an untraditional partner. To them, all they care about is their bottom line. … They want [cooking demonstrations] and they think it attracts folks to the stores. We've been basically doing it because they want it, but our goal was to get them to make some retail changes, like having fruits and vegetables more up front, taking off their liquor and cigarette ads and replacing them with fruit and vegetable ads.
Interviewees consistently described SNAP-Ed efforts in the retail sector as complex, nonlinear, time- and resource-intensive, and challenging, yet exciting when visible changes were made. However, the challenge of competing priorities and limited resources was also described in other settings, including schools. In the next excerpt, the interviewee describes the challenges of working with a school that had limited resources and a new point person with which to partner:
There was a turnover in the district kitchen in [X] and while the person that took the job was very interested in it and he said, “Yes, I'm all for it,” we can't seem to get another moment of his time to say, “Here are some ideas, would you go with it?” . . . It comes down to money. It's expensive for them to buy local and have fresh [produce], because it has to be cleaned. It has to be wrapped up and handled, sent out to each school, a lot of the fresh things don't sustain. The schools themselves are overwhelmed. Like, 1 of the schools, the kitchen staff works crossing guard duty before they go into the kitchen for lunch, and then they go back out and do the crossing guard again, and so they don't really have time.
Nevertheless, some leaders described successes and the impact an individual can have. In the next excerpt, the leader describes working with 1 foodservice director who facilitated school partnerships:
So, the district got a new foodservice director who was just super gung-ho. She even set up a meeting for me with all of these principals. I went from having 1 school that let me teach 1 teacher's class to having 5 or 6 schools all at 1 time because of the help of the foodservice director, which was phenomenal.
Overall, leaders described many factors critical to developing and maintaining partnerships.
Context
Leaders from rural LHDs reported distinct planning, implementation, and evaluation considerations related to funding, geography, population density, staffing, and politics.
Rural programs’ limited funding influenced programming
Those working with limited budgets reported constraints in conducting their work. A number of leaders reported focusing heavily on financial feasibility when planning. One reported staying away from retail interventions:
We're small and we don't have a lot of money. So, I feel like we shied away from those [retail interventions] … People … told me that would just be way too challenging to take on with … the small amount of money we have, and our staffing that we have, too.
Another leader reported focusing on a single intervention to avoid diluting impact:
… Really focusing on 1 PSE strategy at a time, because I think, being in a small county, we're all trying to do so much, and so often the mission gets kind of watered down … narrowing our approach, and really taking just 1 thing at a time to focus on, because we do have limited time and limited resources.
Funding also influenced evaluation. Some rural and small county programs described financial limits on their ability to conduct evaluation:
Yes, we're looking at doing kind of additional evaluation on 1 to 2 of our programs but we're challenged because we don't have the funding per se to get to parcel it out. But we'd like to do more around evaluation so we can see the effectiveness of a couple of the more innovative efforts that we're doing.
Funding also influenced access to trainings, particularly those requiring travel.
Rural settings challenge implementation of certain interventions and evaluation tools
Local leaders working in rural settings reported that interventions common in more urban settings did not apply to their environments. Discussing a program called Safe Walk to School, which encouraged physical activity by encouraging children to walk to school, 1 leader said:
Because we are a rural community, the kids don't live within walking distance of the school. Most of them have … a 20-minute car ride because they're 15 miles away. So, the walking to school event has been hard to figure out.
Another leader reported difficulty convening rural communities:
It's really, really hard work to get the community together … maybe in places where neighborhoods are more cohesive and less brawly-sprawly, it might be easier, but our residents aren't necessarily cohesive by blocks or neighborhoods.
Yet another said that available evaluation tools might not fit the rural context:
It's not always an easy fit to figure out how to use some of the assessment tools and the models that the US Department of Agriculture comes up with and roll them out in a sparsely populated large geographic area.
Rural leaders must carefully consider local politics
A few interviewees shared how they considered politics in conducting their work. One said:
The majority of the population in these counties, I think I can openly say, are very conservative. Sometimes you run against a lot of barriers to making changes that are especially like public health approaches … we've got to get really creative on how you present information, on how you partner.
Another leader said:
I'll just give you a specific example … just hearing Let's Move and then it's associated with Michelle Obama was really just—it was blown off the table, not going to happen. You have to be really careful about who you approach and how you approach it.
Finally, a leader discussed limited opportunities to establish relationships with political leaders:
We walk a fine line because we only have really 1 city council, unlike our neighboring counties. … If they get off to a bad start with their city council, there's always another city that they can go to … but we really only have 1 city where 85% of everything occurs in the county. We walk a fine line with our approach and the strategy that we use.
In rural areas, politics had an important role in determining acceptable programs, framing interventions, and the number of opportunities an LHD might have to engage partners.
Hiring, retention, and capacity were important challenges in rural settings
Local leaders in rural settings highlighted myriad staffing challenges, including a limited hiring pool and an inability to offer competitive salaries:
Staffing challenges for small communities is hard. For one, you have a very small hiring pool … and if you lose 1 out of your team, it's like you just cut off 2 of your feet.
Referring to the SNAP-Ed contract, another leader stated:
That was 1 of the issues, that we don't get enough funding to secure staff that have the expertise needed. It's a complicated contract. It's not an easy contract to understand and do, and as to some of the activities, you have to have a lot of skills. To be able to attract that kind of workforce, you've got to pay, and you've got to have benefits.
Staff retention was also a challenge. Interviewees described how qualified people eventually moved on to better jobs. Some reported hiring part-time staff as a way to manage a limited budget or because of the inability to hire full-time employees, which actually compounded retention issues and affected the program's ability to work effectively with partners. An interviewee described the impact of having part-time staff implementing SNAP-Ed curricula in schools:
With the limited budget, all of our staff, actually besides me and [X], are all part-time … and we have a really fast turnaround in terms of our staff, so it makes it difficult to keep the relationships with some of the schools, because they see this 1 teacher coming and doing the classes, and then all of a sudden they get a full-time job or they move somewhere else, and then we have to bring in a new person, so the teachers and the students have to get used to this new person …
Staffing also sometimes limited their ability to embrace new state priorities:
Being a smaller county … remember that when they put out these big ideas and they do these big things—to remember that some of us can't. We don't have the capacity to do that.
Rural leaders described a challenging cycle: a struggle to hire, which led some to hire part-time staff, high turnover, the subsequent limited ability to meet work demands while being short-staffed, and finally, a return to the long, resource-intensive hiring process.
Local politics, geography, population density, and staffing were important considerations that uniquely shaped the responses of rural leaders regarding implementation of complex nutrition and obesity prevention programs.
Support
Local leaders of the full range of agencies surveyed, large and small, urban and rural, requested several types of support to help them conduct programs, including assistance in creating partnerships and cultivating buy-in, particularly with nontraditional partners; marketing and communication; subcontracting; leadership training; and SNAP-Ed training for new staff.
Local leaders requested support for planning
Leaders requested help with focusing their efforts, having additional time for planning, access to work plan samples, and training on partnership for planning. For example, 1 leader requested more help from the guiding agency on focusing efforts:
It shouldn't just be reducing the problems of obesity. You have to be more specific than that. How are we going to be increasing access to fruits and vegetables in needy communities? How are we going to be increasing opportunities for physical activity? Defining those strategies, setting priorities, saying, “Okay, see … this is based on our research. This is not a good [return on investment] doing community gardens, so you cannot get funded for that.” It shouldn't all be like, “Oh, you can do anything you want.” It should be like, “Okay, we have limited resources in dollars, and this is where we're going to really focus our efforts on, because these are the things that are effective.”
Local leaders requested support for implementing integrated interventions
Local leaders cited a need for more support for PSE change interventions in the form of building staff capacity, strengthening the ability to identify effective interventions and opportunities for conducting these interventions, and integrating direct education with PSE efforts. One leader noted the need for
helping our staff to understand what PSE work really means and looking for those opportunities and providing them adequate training and confidence.
Another requested support to understand what PSE work looked like, particularly for staff members primarily focused on nutrition education:
The staff who have for years and years and years done the nutrition [education] out in the community, they have a really hard time seeing PSE in their work. If there could be more PSE development or PSE trainings to help staff who do nutrition [education] to figure out, to understand what PSE could mean in their work and how it could look, that would be really helpful.
Others reported a need for technical support in the form of step-by-step guidance from the state health department to implement environmental change work:
I think with the retail work, it is nice to see a couple of photos of after shots of stores—that's fantastic—but I want to know, how do I do it? What are the steps? How do I get in there? How do I make that relationship? How do I talk to that retailer? What is the expected time line? They're telling us, “It can take X number of years,” and that's really nice to know, because otherwise, when you don't know, and you're thinking you're going to be able to do this in 6 months, it feels like a fail.
Local leaders asked for support with evaluation
Local leaders frequently and consistently requested more technical step-by step support for evaluation, access to technical experts, clarification about what constituted good evaluation, basic and advanced training on evaluation, and guidance about how to communicate with external evaluators. One said:
We really do need a robust evaluation plan. If there was someone that could walk you through the process of how to do that, and by walk you through, I mean literally hold our hand. Don't give me a training and that's it.
Another asked for training encompassing both quantitative and qualitative evaluation:
Even everything from 101, really basic, like, “What is formative evaluation? What is process evaluation? Why do we collect these numbers? Why do we collect this information?” That would be really helpful all the way up to more advanced stuff about trainings and how we can capture qualitative stuff, how we can capture the stories that we hear, and suggestions on ways to utilize that in a more formal way.
Rural LHDs requested specific training and support
Rural leaders described a lack of fit with some types of support and requested tailored training would benefit them. One described a training held for urban and rural local health departments:
They could have been talking about Mars and it was not anything that was really helpful … anything that we could do to increase skill set, but then also customize it to small rural counties, would be really helpful.
Another rural leader requested more support to understand what was feasible given her funding level:
[I want someone to tell me] if you're getting this amount of funding, you should be working on these 3 or 4 things at minimum … what it is that I'm exactly supposed to be doing at my level of funding and with the number of staff that I have.
Rural sites also requested more online training options, more advanced notice of trainings held in distant locations, and increased travel budgets to better support travel to centralized trainings.
Discussion
The SNAP-Ed leaders at public health departments across California provided a snapshot of the processes, successes, and challenges of planning, implementing, and evaluating nutrition education and obesity prevention programs shortly after adopting these integrated intervention models. Most saliently, this study suggests that although the importance of environmental change interventions is well-established in the literature and accepted by researchers and practitioners, local leaders grappled with how to introduce, implement, and integrate these interventions at both the conceptual and practical levels.
Conceptually, leaders struggled to integrate direct education with environmental change interventions, compartmentalizing the 2 approaches. Bernardo et al challenged entrenched and oversimplified dichotomies in debates about causes of and solutions to obesity, arguing for a reframing that emphasizes reciprocal interaction between the environment and individuals.
15
Similarly, Brownell et al16
asserted that understanding connections between personal responsibility and environments is constructive. Helping local leaders bridge the divide between educational and environmental change approaches to obesity prevention may lead to more nuanced, synergistic, and ultimately effective approaches.The current authors learned that planning, implementing, and evaluating complex nutrition education and obesity prevention programs were often complicated where they might theoretically seem simple or straightforward, and variable where they might seem to be standardized. Decision-making for planning was 1 such complex area. The finding that health departments use markedly varied decision-making processes, weighing a host of practical, contextual factors (eg, funding, staff capacity) with little reference to research evidence, was consistent with research examining decision-making and planning in public health programs.
6
, 17
, 18
A systematic review17
examining public health decision-making processes found that practitioners often made decisions and sought justification after the fact. Like the current study, the review found that feasibility, funding pressure, local politics, internal program data, and previous experience informed decision-making; it noted that consultation with experts, common sense, and the opinions of key personnel were used as proxies for evidence-based decision-making owing to competing considerations, limited staff time and capacity, being underresourced, and practitioners finding the research irrelevant. Thus, although the National Academy of Medicine put forth a framework for decision-making and recommended a registry for keeping decision-makers current regarding the literature, evidence indicated that it was not simply the lack of access to literature or a framework that inhibited integration of evidence into decision-making.18
Rather, in conjunction with the cited research, this study suggests the need to support programs and practitioners better by understanding their decision-making process and their underlying reasons for neglecting to incorporate scientific evidence. Moreover, because the current study suggests that implementers often do not seem to integrate both practical information and research evidence, future studies might explore whether a tool or resource for weighing and integrating both real-time factors and the evidence base could improve and standardize the decision-making process and potentially improve outcomes.Local leaders requested additional resources and skill building to help them strengthen partnerships. Although they characterized the process of establishing, building, and maintaining partnerships as important, they highlighted multiple challenges. Other studies underscored the hard work and importance of partnership, and suggested that it requires clear and frequent communication, a clarification of shared roles and objectives, governance, strong leadership, transparency, and accountability once trust was established.
19
, 20
Multiple studies concluded that partnerships are major contributors to the success of obesity prevention programs.19
, 21
, 22
The current researchers learned that establishing partnerships with nontraditional partners requires skills, time, and energy, a finding corroborated by Gantner and Olson,21
who evaluated the capacity of public health professionals to implement similar types of programs and found that establishing such partnerships constituted a novel skill set for public health professionals. Managing expectations of funders and practitioners related to the demands, potential pitfalls, and potential benefits of establishing partnerships is critical.The current study highlighted a set of practical skills needed to deliver PSE interventions better, such as prioritizing effective interventions, partnership development, and program evaluation. Nevertheless, other studies emphasized the concurrent need to build public health professionals’ familiarity and engagement with the public policy process more generally.
22
, 23
, 24
, 25
, 26
In 1 specific example, the majority of 84 SNAP-Ed staff rated a training focused on PSE skills and partnership development as very useful or moderately useful, perhaps signaling an interest in building such skills.27
Thus, both knowledge and practical skills are needed to implement PSE interventions better.Rural communities experience higher rates of obesity than urban communities do, yet much of the evidence that informs policy and environmental change strategies is based largely on urban and suburban studies.
28
The current findings indicated that rural programs either avoid specific obesity prevention strategies or adapt strategies that are effective in urban settings but are incompatible with the rural setting. A literature review examining 12 policy and environmental change strategies recommended by the CDC to influence physical activity found that 3 of the recommended strategies were not implemented in any of the 26 identified studies. These strategies were related to school location, improving public transportation, mixed-use zoning, enhancing personal safety, and traffic safety in areas where people could be physically active. They instead focused on 7 strategies that seemed more feasible in rural settings (eg, requiring physical education in schools, increasing the amount of physical activity in physical education programs in schools, increasing opportunities for extracurricular physical activity, reducing screen time in public service venues, improving access to outdoor recreational facilities, enhancing infrastructure supporting bicycling, enhancing infrastructure supporting walking, support for locating schools within easy walking distance of residential areas, and improving access to public transportation).28
Studies exploring barriers to implementing specific strategies, including PSE, in the rural setting identified factors such as staff turnover; lack of staff buy-in; and limited funding, lack of community buy-in, political will, or community support.28
, 29
A few studies suggested that partnership is critical in rural settings,29
, 30
indicating that support focused on partnership might allow rural programs to stretch limited resources. A study specifically examining implementing PSE in rural settings found a need for more education regarding how to implement PSE in rural settings.29
Overall, the current findings suggested that rural programs confront unique challenges. Further work might explore whether providing context-sensitive interventions and tailored support for implementation in rural settings facilitates success.This study had limitations. The researchers did not interview other stakeholders, such as non–public health partners, and relied solely on interviews and panel interviews to understand local health department processes. The researchers were unable to distinguish the unique challenges of rural programs from those of small budget programs. As a qualitative study conducted within the unique context of SNAP-Ed in California, the findings of this study may not be generalizable to other SNAP-Ed programs or similar efforts funded by others. However, the authors were able to address some limitations by interviewing local leaders across California to the point of saturation. Panel interviews allowed preliminary findings to be tested and multiple coders improved the reliability of findings. Although the findings may not be generalizable, the lessons learned may be transferable to SNAP-Ed programs as well as other programs and practitioners working to improve nutrition and physical activity in low-income California communities and beyond.
Implications for Research and Practice
This study illuminated process and practice among CDPH-funded SNAP-Ed programs that were relatively new to planning, implementing, and evaluating PSE interventions. The findings highlight specific challenges confronted by local leaders, such as enthusiasm for new approaches coupled with a need for additional capacity, inconsistencies in planning and implementation processes, and differences in experiences between programs in rural and urban areas. This information suggests new ways to support local agencies in planning, implementing, and evaluating complex interventions.
These findings are particularly useful for others in the early stages of introducing complex nutrition education and obesity prevention interventions and are broadly informative for programs and funders working in this area.
Planning and decision-making require local leaders to incorporate broad swathes of real-time and evidence-based data. Resources in the literature supporting complicated decisions and allowing practitioners to access and digest evidence easily are critical. Given the disparate planning processes described in this study, the ideal form of support would articulate a standard set of considerations for planning while allowing for variation in local decision-making processes. Guidance for programs that receive technical support could be informed by an understanding of local decision-making and planning processes.
In addition to support for determining which interventions to implement, local leaders requested ongoing, step-by-step guidance for how to implement interventions, particularly PSEs. This study found that partnership development requires significant effort and skill to be successful. Findings from other studies suggested that this investment in partnerships is worthwhile because of their potential to achieve enhanced effects on desired outcomes. Obesity prevention and nutrition programs without adequate resources and skills to cultivate partnerships may miss opportunities to leverage funds, synergize efforts, and ultimately increase families’ access to healthy foods.
Basic evaluation training for local leaders could strengthen evaluation capacity and confidence; they would benefit from reporting requirements that monitor performance while providing them with timely and useful information about local program impact. Improved linkages among planning, implementation, and evaluation, such that planning is well-informed by specific and local evaluation data, or evaluation conducted with measurement of key program processes and/or outcomes, would also strengthen programs.
The researchers affirm that context matters. Rural programs reported experiences distinct from their urban counterparts, which implies a need for specific support, resources, and interventions in different settings. Local politics, geography, population density, and staffing were important considerations that uniquely shaped the responses of rural leaders regarding implementation of complex nutrition and obesity prevention programs. Training implementers to identify appropriate interventions in varied contexts and building capacity to engage in partnerships may lead to better use of resources and higher program impact.
Studies of implementation and process in rural and urban settings are critical for understanding current practice and identifying ways to improve program delivery. There is a need for more examination of how practice is distinct from, and often lags behind, what is deemed effective in the literature.
Appendix. Supplementary data
References
- Preventing childhood obesity: health in the balance: executive summary.J Am Diet Assoc. 2005; 105: 131-138
- Accelerating Progress in Obesity Prevention: Solving the Weight of the Nation.The National Academies Press, Washington, DC2012
- Creating healthy food and eating environments: policy and environmental approaches.Annu Rev Public Health. 2008; 29: 253-272
- Evidence-based intervention in physical activity: lessons from around the world.Lancet. 2016; 380: 272-281
- Process evaluation outcomes from a global child obesity prevention intervention.BMC Public Health. 2014; 14: 757
- Strengthening implementation and utilization of nutrition interventions through research: a framework and research agenda.Ann N Y Acad Sci. 2014; 1332: 39-59
- Community-based interventions for obesity prevention: lessons learned by Australian policy-makers.BMC Res Notes. 2012; 5: 20
- A mixed methods protocol for developing and testing implementation strategies for evidence-based obesity prevention in childcare: a cluster randomized hybrid type III trial.Implement Sci. 2017; 12: 1-10
US Department of Agriculture Economic Research Service. Funding allocations. FY 2018 final state allocations. https://snaped.fns.usda.gov/program-administration/funding-allocations. Accessed December 17, 2018.
- Restructuring a state nutrition education and obesity prevention program: implications of a local health department model for SNAP-Ed.J Public Heal Manag Pract. 2017; 23: e28-e36
US Department of Agriculture Supplemental Nutrition Assistance Program–Education (SNAP-Ed) factsheet. https://snaped.fns.usda.gov/snap/SNAP-Ed%20Factsheet%20_August%202016.pdf. Accessed March 22, 2019.
- Nutrition Education: Linking Theory, Research and Practice.3rd edition. Jones and Bartlett Publisher, Subury, MA2011
Centers for Disease Control and Prevention. NCHS Urban-Rural Classification Scheme for Counties. National Center for Health Statistics. https://www.cdc.gov/nchs/data_access/urban_rural.htm#2013_Urban-Rural_Classification_Scheme_for_Counties. Accessed March 22, 2019.
- Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?.BMJ. 2001; 322: 1115-1117
- Patchy progress on obesity prevention: emerging examples, entrenched barriers, and new thinking.Lancet. 2015; 385: 2400-2409
- Personal responsibility and obesity: a constructive approach to a controversial issue.Health Aff. 2010; 29: 379-387
- The use of research evidence in public health decision making processes: systematic review.PLoS One. 2011; 6: e21704
Kumanyika SK, Parker L, Sim LJ, editors. Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making. Washington, DC: The National Academies Press; 2010.
- The value of partnerships in state obesity prevention and control programs.Health Promot Pract. 2012; 13: 222-229
- A public-private partnership model for obesity prevention.Prev Chronic Dis. 2009; 6: A110
- Evaluation of public health professionals’ capacity to implement environmental changes supportive of healthy weight.Eval Program Plann. 2012; 35: 407-416
- Assessing competencies for obesity prevention and control.J Law Med Ethics. 2009; 37: 37-44
- For the Public's Health: Revitalizing Law and Policy to Meet New Challenges.National Academies Press, Washington, DC2011
- A framework for physical activity policy research.J Phys Act Health. 2006; 3: S20-S29
- Legal preparedness for obesity prevention and control: a framework for action.J Law Med Ethics. 2009; 37: 15-23
- Policy, systems and environmental approaches for obesity prevention: a framework to inform local and state action.J Public Health Manag Pract. 2016; 19: 773-779
- A training approach to build capacity in policy, systems, and environmental change for nutrition program staff.J Nutr Educ Behav. 2016; 48: S29
- Physical activity–related policy and environmental strategies to prevent obesity in rural communities: a systematic review of the literature, 2002–2013.Prev Chronic Dis. 2016; 13150406
- Barriers to delivering policy, systems and environmental changes in rural communities.Nutrients. 2018; 10: 1012
- Disparities in obesity among rural and urban residents in a health disparate region.BMC Public Health. 2014; 14: 1-8
Article info
Publication history
Accepted:
February 19,
2019
Footnotes
Conflict of Interest Disclosure: The authors have not stated any conflicts of interest.
Identification
Copyright
© 2019 Published by Elsevier Inc. on behalf of Society for Nutrition Education and Behavior.
User license
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0) | How you can reuse 
Elsevier's open access license policy
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0)
Permitted
For non-commercial purposes:
- Read, print & download
- Redistribute or republish the final article
- Text & data mine
- Translate the article (private use only, not for distribution)
- Reuse portions or extracts from the article in other works
Not Permitted
- Sell or re-use for commercial purposes
- Distribute translations or adaptations of the article
Elsevier's open access license policy
