Advertisement
Research Article| Volume 49, ISSUE 7, SUPPLEMENT 2, S177-S185.e1, July 2017

Download started.

Ok

Implementing a WIC-Based Intervention to Promote Exclusive Breastfeeding: Challenges, Facilitators, and Adaptive Strategies

      Abstract

      Objective

      Understand factors that contributed to the implementation of a successful multicomponent intervention to promote exclusive breastfeeding (EBF) within Special Supplemental Nutrition Program for Woman, Infants, and Children (WIC) clinics.

      Design

      Qualitative study of staff implementers' experiences using implementation status reports, facilitated group discussion immediately after implementation, and WIC administrative data.

      Setting

      WIC staff from 12 clinics participated in an EBF Learning Community composed of 8 intervention trainings and ongoing support from trainers and peers.

      Participants

      A total of 47 WIC staff including 11 directors, 20 other administrators, 8 nutritionists, and 6 peer counselors.

      Intervention

      A WIC-integrated EBF promotion initiative, supported through a Learning Community, composed of prenatal screening, tailored trimester-specific counseling, and timely postpartum follow-up.

      Phenomenon of Interest

      Challenges and facilitators to implementation within clinics.

      Analysis

      Iterative qualitative analysis using directed, emergent, and thematic coding.

      Results

      Implementation experiences were characterized by (1) perceived benefits of implementation, including improved EBF knowledge and counseling confidence among staff; and (2) managing implementation, including responding to challenges posed by clinic settings (resources, routine practices, values, and perceptions of mothers) through strategies such as adapting clinic practices and intervention components.

      Conclusions and Implications

      Implementation was shaped by clinic setting and adaptive strategies. Future WIC interventions may benefit from formal consideration of intervention fit with local clinic setting and allowable adaptations.

      Key Words

      Introduction

      Breastfeeding (BF) is associated with numerous benefits to infant and maternal health
      • Ip S.
      • Chung M.
      • Raman G.
      • et al.
      Breastfeeding and maternal and infant health outcomes in developed countries.
      and leading public health organizations recommend exclusive breastfeeding (EBF) for about 6 months to maximize health benefits.
      • Eidelman A.I.
      • Schanler R.J.
      • Johnston M.
      • et al.
      Breastfeeding and the use of human milk.
      The Special Supplemental Nutrition Program for Woman, Infants, and Children (WIC) prioritizes BF promotion through provision of the enhanced BF food package, BF aids such as breast pumps, as well as BF education and counseling.
      Despite these coordinated activities, 2014–2015 National Immunization Survey data

      National Immunization Survey, Centers for Disease Control and Prevention. Rates of any and exclusive breastfeeding by socio-demographics among children born in 2013. https://www.cdc.gov/breastfeeding/data/nis_data/rates-any-exclusive-bf-socio-dem-2013.htm. Accessed January 11, 2017.

      show that whereas a majority of mothers participating in WIC initiated BF (74.1%), less than 15.7% were EBF through 6 months, which is below both the national rate of 22.3% and the rate among income-eligible non-WIC participants (29.1%). This trend suggests that WIC mothers may face unique challenges maintaining EBF after initiation.
      The WIC program is a promising intervention setting given its wide reach and capacity to deliver effective components of successful BF promotion interventions such as lay and professional support, education, and counseling spanning pregnancy and the first year postpartum.
      • Renfrew M.J.
      • McCormick F.M.
      • Wade A.
      • Quinn B.
      • Dowswell T.
      Support for healthy breastfeeding mothers with healthy term babies.
      • Patnode C.D.
      • Henninger M.L.
      • Senger C.A.
      • Perdue L.A.
      • Whitlock E.P.
      Primary care interventions to support breastfeeding: Updated evidence report and systematic review for the US Preventive Services Task Force.
      • Chung M.
      • Raman G.
      • Trikalinos T.
      • Lau J.
      • Ip S.
      Interventions in primary care to promote breastfeeding: an evidence review for the U.S. Preventive Services Task Force.
      Although a growing number of interventions to improve EBF were conducted with WIC-eligible and low-income populations,
      few were based within the WIC care setting.
      • Reeder J.A.
      • Joyce T.
      • Sibley K.
      • Arnold D.
      • Altindag O.
      Telephone peer counseling of breastfeeding among WIC participants: a randomized controlled trial.
      • Hildebrand D.A.
      • McCarthy P.
      • Tipton D.
      • Merriman C.
      • Schrank M.
      • Newport M.
      Innovative use of influential prenatal counseling may improve breastfeeding initiation rates among WIC participants.
      Integration into the WIC care setting may allow initiatives to build on existing resources
      • Reeder J.A.
      • Joyce T.
      • Sibley K.
      • Arnold D.
      • Altindag O.
      Telephone peer counseling of breastfeeding among WIC participants: a randomized controlled trial.
      • Hildebrand D.A.
      • McCarthy P.
      • Tipton D.
      • Merriman C.
      • Schrank M.
      • Newport M.
      Innovative use of influential prenatal counseling may improve breastfeeding initiation rates among WIC participants.
      and produce broader and more sustained benefit.
      • Chung M.
      • Raman G.
      • Trikalinos T.
      • Lau J.
      • Ip S.
      Interventions in primary care to promote breastfeeding: an evidence review for the U.S. Preventive Services Task Force.
      These findings underline an urgent need for studies that intend to identify challenges and facilitators to integrating EBF promotion interventions within WIC. Implementing programs within organizations is a complex process that involves adoption and support from administrators as well as coordinated and interdependent effort by a variety of organizational staff.
      • Weiner B.J.
      • Lewis M.A.
      • Linnan L.A.
      Using organization theory to understand the determinants of effective implementation of worksite health promotion programs.
      Research findings from implementation of health programs in diverse organizational settings highlight the need to explore not only indicators of implementation such as fidelity and reach measures but also organizational factors that shape the implementation process and sustainability within local contexts.
      • Weiner B.J.
      • Lewis M.A.
      • Linnan L.A.
      Using organization theory to understand the determinants of effective implementation of worksite health promotion programs.
      • Nickel N.C.
      • Taylor E.C.
      • Labbok M.H.
      • Weiner B.J.
      • Williamson N.E.
      Applying organization theory to understand barriers and facilitators to the implementation of baby-friendly: a multi-site qualitative study.
      • Isbell M.G.
      • Seth J.G.
      • Atwood R.D.
      • Ray T.C.
      A client-centered nutrition education model: lessons learned from Texas WIC.
      In 2014, New York State (NYS) WIC piloted a WIC-based intervention, You Can Do It (YCDI), to promote EBF in 12 WIC clinics. You Can Do It aimed to enhance promotion of EBF through staff training and support to deliver prenatal screening, individually tailored and trimester-specific counseling, and timely postpartum follow-up to enrolled mothers. An outcome evaluation focusing on EBF at 7, 30, and 60 days demonstrated that the intervention was effective.
      • Edmunds L.S.
      • Lee F.F.
      • Eldridge J.D.
      • Sekhobo J.P.
      Outcome evaluation of the You Can Do It initiative to promote exclusive breastfeeding among women enrolled in the New York State WIC program by race/ethnicity.
      This study aimed to identify factors that affected implementation of this successful multicomponent intervention within WIC clinics. In particular, this study qualitatively explored challenges and facilitators to implementation using data from staff implementer experiences.

      Methods

      Intervention Setting and Design

      You Can Do It, an initiative developed by Vermont WIC,
      • Bortree L.
      • Decher L.
      • Flynn K.
      Supporting Long Term Breastfeeding With the New WIC Food Packages—You Can Do It / WIC Can Help.
      aimed to improve EBF among WIC participants by providing training and support for BF promotion protocols
      • Bortree L.
      • Decher L.
      • Flynn K.
      Supporting Long Term Breastfeeding With the New WIC Food Packages—You Can Do It / WIC Can Help.
      • Gill S.L.
      • Reifsnider E.
      • Lucke J.F.
      • Mann A.R.
      Predicting breast-feeding attrition: adapting the Breast-feeding Attrition Prediction Tool.
      • Janke J.R.
      Development of the Breast-Feeding Attrition Prediction Tool.
      • Hawkins R.P.
      • Kreuter M.
      • Resnicow K.
      • Fishbein M.
      • Dijkstra A.
      Understanding tailoring in communicating about health.
      (Table 1) to supervisory, professional, and paraprofessional clinic staff. You Can Do It was implemented in 12 WIC clinics with diverse characteristics across NYS from July, 2014 through October, 2015 (Table 2). These clinics agreed to commit staff time and resources to make changes within clinic systems to support implementation. This study protocol was approved by the institutional review board of the NYS Department of Health.
      Table 1Components of the You Can Do It Initiative by Prenatal (PN) and Postpartum (PP) Sequencing
      TimeIntervention Components
      Recruitment of eligible women in WIC clinics
       First trimester
      • Enroll eligible pregnant women. Eligibility criteria:
        • Aged 18 y or older
        • Intend to breastfeed or undecided
        • Read and understand English or Spanish
      Contacts with enrolled mothers
       First trimester
      • Administer modified BAPT
        • Gill S.L.
        • Reifsnider E.
        • Lucke J.F.
        • Mann A.R.
        Predicting breast-feeding attrition: adapting the Breast-feeding Attrition Prediction Tool.
        • Janke J.R.
        Development of the Breast-Feeding Attrition Prediction Tool.
        • Individual face-to-face appointment with nutritionist
        • Assess PN mother's attitudes, confidence, and support to breastfeed using 26 scored items
       Second trimester
      • Provide tailored counseling using results from BAPT
        • Individual face-to face appointment with nutritionist
        • Provide counseling addressing areas identified by BAPT through combination of personalization, feedback, and content matching
          • Hawkins R.P.
          • Kreuter M.
          • Resnicow K.
          • Fishbein M.
          • Dijkstra A.
          Understanding tailoring in communicating about health.
       Third trimester
      • Conduct targeted group discussion
        • Group setting conducted by nutritionists and PCs
        • Prepare PN mothers for hospital experience, including education about hospital practices that support breastfeeding and communication of feeding plan to birth support team
       1–3 d PP
      • Originate first PP contact
        • Telephone call with nutritionist
        • Early assessment of hospital experience, breastfeeding and recognition of infant feeding cues
        • Early affirmation to improve confidence
        • Offer support and referrals
       3–5 d PP
      • Conduct second PP contact
        • Telephone call with nutritionist or contact/call with PC
        • Follow-up assessment of breastfeeding and baby behaviors
        • Open-ended questions to assess breastfeeding, educate, and prevent introduction of formula
       2–4 wk PP
      • Certify breastfeeding dyad
        • Individual face-to-face appointment with nutritionist
        • Conduct certification, provide appropriate food package and follow up on breastfeeding support
      Other intervention components
       First to second trimesters
      • Case conferencing
        • Nutritionist, breastfeeding coordinator, and PC develop coordinated/tailored breastfeeding care plan
        • Required when BAPT score ≤ 20
       Ongoing
      • Nutritionist continuity
        • Recommend all mothers meet with same nutritionist throughout PN and PP periods
        • Suggested to improve rapport and care continuity
      BAPT indicates Breastfeeding Attrition Prediction Tool; PC, peer counselor; PN, prenatal; PP, postpartum; WIC, Special Supplemental Nutrition Program for Women, Infants, and Children.
      Table 2Characteristics of WIC Clinics and Performance Improvement Teams Participating in the You Can Do It Initiative in New York State
      Count
      Characteristics of WIC clinics (n = 12)
      Clinic caseload
      Infant certification in federal fiscal year 2014 (New York State WIC administrative data)
       Small (<500 infants)4
       Medium (500–1,000 infants)6
       Large (>1,000 infants)2
      Clinic location
      Categorization is based on the 2010 primary and secondary Rural Urban Commuting Area Codes35
       Urban10
       Non-urban2
      Clinic setting
       Medical center7
       Non-medical center
      Non-medical centers include local health departments, community centers, and standalone office buildings.
      5
      Breastfeeding coordinator time spent on breastfeeding activities
       ≥40% h6
       <40% h6
      Composition of 12 teams by job title (n = 47)
       WIC director11
       Breastfeeding coordinator12
       Peer counselor coordinator4
       Other administrator4
       Nutritionist or nutrition assistant8
       Peer counselor6
       Unspecified2
      WIC indicates Special Supplemental Nutrition Program for Women, Infants, and Children.
      a Infant certification in federal fiscal year 2014 (New York State WIC administrative data)
      b Categorization is based on the 2010 primary and secondary Rural Urban Commuting Area Codes

      US Department of Agriculture, Economic Research Service. Rural-urban commuting area codes. https://www.ers.usda.gov/data-products/rural-urban-commuting-area-codes/documentation/. Accessed May 12, 2017.

      c Non-medical centers include local health departments, community centers, and standalone office buildings.
      Central to YCDI implementation in NYS was the formation of a Learning Community,
      • Institute for Healthcare Improvement
      The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement.
      a model for collaborative learning to promote quality improvement within organizations, by the NYS WIC Training Center contractor. The Learning Community brought together staff from participating clinics to learn from intervention trainers, experienced YCDI implementers from Vermont WIC, and the practical experiences of participating peers over an intensive and ongoing intervention training process involving a series of Learning Community Sessions and Action Periods.
      • Institute for Healthcare Improvement
      The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement.
      Each participating WIC clinic established a performance improvement team, referred to here as teams, to represent its clinic in the Learning Community Sessions. Teams were intended to represent supervisory, professional, and paraprofessional staff and were composed of up to 4 members from each clinic (Table 2). Teams attended a total of 8 joint Learning Community Sessions (ie, 2 in person and 6 via webinar) that addressed, for each intervention component, background and practical information, necessary implementation skills, and technical assistance. In Learning Community Sessions, teams developed implementation plans specific to their clinic with guidance from trainers and other peer teams. Teams also discussed implementation progress with one another in Learning Community Sessions. In between each Learning Community Session were Action Periods, in which teams returned to their local clinics to train other WIC staff and carry out planned organizational changes.

      Data

      The data presented in this paper were provided by staff implementers in 3 ways: (1) clinic progress reports presented by teams to trainers and peers during Learning Community Sessions, (2) facilitated group discussion with a convenience sample of implementing staff after the intervention, and (3) NYS WIC administrative data.

      Learning Community Session status reports

      Each team presented implementation status reports to trainers and peers at 4 Learning Community Sessions throughout the intervention period (ie, August, 2014, October, 2014, December, 2014, and January, 2015). Status reports addressed implementation challenges, facilitators, and accomplishments. Although all status reports were presented orally at Learning Community Sessions, only 1 (December, 2014) was recorded by intervention trainers and transcribed verbatim. All other status reports were received as written documents.

      Follow-up facilitated group discussion

      The second set of data was collected in a follow-up facilitated group discussion with implementing WIC staff in December, 2015 immediately after the implementation period. All teams were invited to participate; 22 team members (15 administrators, 5 nutritionists, and 2 peer counselors) from 10 clinics attended. Staff members were placed into 6 groups. Three intervention trainers rotated groups to facilitate discussion of implementation challenges, facilitators, resources, and training needs. Each group created written summaries of their discussion, which were used for this analysis. Trainers then brought the groups together to facilitate discussion as an entire group, asking probing questions to expand written responses and confirm interpretation. Researchers composed field notes during the discussion that were also used during this analysis.

      Administrative data

      The researchers used administrative data to assemble information about the organizational context of each WIC clinic during the intervention period (October, 2014 to September, 2015). Reviewed data included clinic staffing levels, physical space, and other resources, as well as previous and current BF promotion activities and goals.

      Data Analysis

      All data were analyzed in text format using Microsoft Word 2016 (Microsoft Corporation, Redmond, WA). All names and locations were replaced with staff titles and location codes. Status reports and administrative data pertaining to a specific clinic were grouped and analyzed before moving on to analysis of the next clinic. In addition, status reports were analyzed sequentially by document date.
      Data were analyzed using directed content analysis,
      • Hsieh H.F.
      • Shannon S.E.
      Three approaches to qualitative content analysis.
      in which concepts from organizational theory,
      • Weiner B.J.
      • Lewis M.A.
      • Linnan L.A.
      Using organization theory to understand the determinants of effective implementation of worksite health promotion programs.
      the YCDI logic model, and the study research question were used as sensitizing concepts in the initial coding phase. Data were coded using a priori labels as well as identification of emergent patterns related to implementation experiences, challenges, and facilitators. Codes were then grouped thematically. Throughout coding, the constant comparative method was used to identify and sort codes and quotations by emergent themes.
      • Miles M.B.
      • Huberman A.M.
      Qualitative Data Analysis: An Expanded Sourcebook.
      All data were analyzed, although no new themes relevant to implementation experiences emerged after coding of data from the eighth clinic.
      The quality and integrity of this analysis were strengthened by the involvement of 2 researchers who coded data independently and then met regularly to discuss and compare findings. Discrepancies in coding were resolved through these discussions. Feedback on preliminary interpretations was also obtained from other members of the research team, intervention trainers, and staff implementers through internal presentation of findings.

      Results

      The WIC staff implementers provided rich descriptions of their experiences implementing YCDI and how implementation unfolded over the course of the initiative. Study findings revealed 2 salient and overarching themes. First, implementation was perceived as benefiting individual WIC staff knowledge and confidence as well as improving clinic support for BF promotion. Second, staff described implementation as a multicomponent process in which challenges posed by the interplay between YCDI intervention components and local clinic settings were managed through strategies such as adapting both clinic practices and intervention components. Themes are presented subsequently.

      Perceptions of Implementation Benefits

      Staff reported several positive impacts of implementing YCDI, such as perceptions of improved communication, teamwork, and confidence to provide BF counseling, as well as the opportunity to identify and address areas for BF improvement. Staff also reported perceptions of improved resources, practices, and attitudes related to BF promotion.

      Perceived improvements in staff communication and teamwork

      Staff descriptions highlighted improved communication, specifically between peer counselors and nutritionists who might not have worked as closely due in part to disparate work schedules and locations. Case conferencing (Table 1) was described as “great for communication” and a process that “encouraged CPA [competent professional authority or nutritionist] and PC [peer counselor] to communicate more.” One team member noted, “Because of case conference, CPAs are more open with peer counselors.”
      Staff described how specific intervention protocols such as including peer counselors in trainings facilitated communication, “validated the PC/CPA team” and promoted “PC and CPA giving the same message.” One peer counselor coordinator described information-sharing between peer counselors and nutritionists in preparation for a YCDI clinic visit: “I give them [nutritionists] the notes from the peer counselor that's had conversations with them [participating mother], so this way they can prepare and they know where they're at with the moms.”

      Perceived improvements related to confidence in counseling

      Staff reported that using the Breastfeeding Attrition Prediction Tool (BAPT)
      • Gill S.L.
      • Reifsnider E.
      • Lucke J.F.
      • Mann A.R.
      Predicting breast-feeding attrition: adapting the Breast-feeding Attrition Prediction Tool.
      • Janke J.R.
      Development of the Breast-Feeding Attrition Prediction Tool.
      and targeted counseling protocol (Table 1) improved counseling confidence by reducing uncertainty regarding appropriateness of topics covered and “gave a focused and targeted purpose to the visit.” One team member described the initiative as “… a benefit because we are given specific instructions to help us better educate our participants about breastfeeding.”
      Staff also described perceived improvements in interactions between staff and mothers. Staff described perceptions of improved “rapport,” a “stronger relationship,” and “more intimate conversation” with mothers and described how “information from BAPT helped CPA draw out more reserved participants, starting point for conversation.”

      Identified areas for improvement

      Staff described how preparation to lead tailored counseling and BF discussion groups provided an opportunity to identify and address gaps in BF knowledge and counseling competency and confidence among clinic staff. For example: “Nutritionist[s] felt nutrition was their thing, not breastfeeding … needed to boost their confidence that they could do it.” As another staff member described in a Learning Community Session:Staff: Because sometimes our staff are not clear about certain things, like how the breast, you know, make milk and we get like really into it because we want them to give them the correct information.Trainer: So it sounds like you've had to do some retraining on breastfeeding information for all of your staff to make sure everyone's knowledgeable.Staff: We went over a lot of different things like skin-to-skin, feeding cues—all of that stuff. We also talked to them in the meeting, because we're also getting ready for the group.
      For some agencies, participation provided an opportunity to examine and address BF values among staff; for example:This proved to be an exceptional experience for our agency and opened our eyes to issues internally that might cause us to not be nearly as effective with breastfeeding support as we can be. … In order to move staff forward who may have personal beliefs or opinions on breastfeeding, [clinic] will be subcontracting with [training agency] to provide a targeted training for our agency staff.

      Perceptions of changes in clinics resources, practices, and attitudes related to breastfeeding promotion

      Several clinics reported changes in their resources, practices, or attitudes related to BF promotion. Perceived improvements in resources included enhanced nutritionist BF counseling skills, as well as enriched partnerships to outreach to prenatal women. Perceived improved practices included protocols to facilitate first-trimester enrollment of prenatal mothers as well as integrating the screening and targeted counseling protocol into routines in sister-clinics. Clinics also described perceptions of “increase[d] enthusiasm for breastfeeding promotion across the whole staff” and “new, revived energy about helping educate moms about breastfeeding.”

      Managing Implementation: Challenges, Facilitators, and Strategies

      Managing implementation thematically weaves together connected descriptions of implementation challenges posed by the interplay between intervention components and the clinic setting as well as the development of strategies to address challenges. The clinic setting refers to clinic resources (eg, staffing levels and time constraints), values, routine practices, and perceptions of local WIC mothers. Strategies to address challenges included building on facilitative clinic settings, adapting clinic practices, and sometimes adapting intervention components. Instances of managing implementation are presented and discussed in depth.

      Balancing clinic flow

      The developers of the YCDI protocol intended enrolled mothers to see the same nutritionist and peer counselor for all pregnancy and postpartum follow-up visits. In addition, YCDI advised clinics to increase the allotted time for tailored counseling from the usual 15-minute counseling visit to 30 minutes to support complete discussion of intervention materials. However, staff described how nutritionist continuity and recommended scheduling were challenged by staff values regarding clinic flow, perceptions of mothers’ attendance, and issues with staffing.
      Maintaining clinic flow and short participant wait times emerged as an important value for staff and a perceived value of mothers, leading to initial resistance to lengthen appointment time. Clinics reported administrative goals to “optimize patient flow” and respect mothers who were “in a rush.” Some teams reported that clinic support staff would schedule over the 30-minute slot to maintain clinic flow:I think people are being respectful of the need for 30 minutes … In the beginning you would schedule 30 minutes and then when they arrived you'd see that somebody [WIC staff] had taken away half the appointment, so that was a challenge.
      Perceptions of mothers as needing to reschedule frequently because of child care and transportation issues were described as a challenge: “Our participants often have transportation problems and will show up when they can … not when they are scheduled.” Staff described how participants usually saw any available nutritionist to afford clinics some flexibility in responding to schedule changes, while maintaining clinic flow. Rescheduling and maintaining the same nutritionist was particularly challenging in light of staffing issues such as position vacancies, as well as vacations (summer period) and staff rotations to other WIC clinics.
      In efforts to balance clinic flow with nutritionist continuity and recommended scheduling, staff described several strategies. Many of these strategies involved creating implementation protocols to support and enable both staff and participants to adhere to the scheduled appointments. Strategies aimed at participants included reminder pamphlets, letters, text messages, and phone calls to mothers. For staff, many clinics developed protocols that capitalized on existing routine meeting times to forecast participant appointments so that nutritionists could prepare and accommodate.Every day at the huddle we do a daily forecast—it's printed and given to all the CPAs and it's also visible for all the other staff to see who's scheduled for the week and even as far as next week—what kind of appointment they're coming for and who enrolled them. So if CPA sees, ‘Oh, I'm not here next Monday,’ they can set it up with another CPA.
      Another strategy involved creating clinic-wide implementation policies to support longer appointments by amending the clinic scheduling system and reviewing the study protocol with those responsible for scheduling:Somebody last check-in [Learning Community Session] said to put names in CAPS in [NYS WIC scheduling system] and we thought that was a great idea, so we did that. And it's been working out fantastic. For 1 thing, it helps us a lot to identify them as soon as they arrive and then it also helps our staff when they're looking for space for appointments to be respectful of the 30-minute appointment and to know that they can't modify that.
      Other clinic supervisors blocked off time and cautioned staff not to rush through the tailored counseling visits: “… they have to use those 30 minutes, have to be prepared and they have to counsel that woman so that she gets all the information she needs with the handouts.”
      A third major strategy involved adapting the intervention to provide continuity of care even if nutritionist continuity could not be maintained. Clinics used a combination of preparing tailored materials for each participant in advance and placing those materials in a centralized location accessible by all staff to facilitate continuity of care. As staff from 1 clinic reported:We found it effective to prepare all of the materials in advance. … We put it all together into a plastic sleeve so that it’s all ready when the mother comes. And also, in case that something happens and the CPA [nutritionist] who prepared this is not there, it's all ready and available for anyone who needs to do it.
      It was common for staff to comment on strategies from other clinics shared at Learning Community Sessions and communicate future implementation strategies. For example, staff members from 1 clinic responded positively to a strategy from their peers:There were some good ideas today that we might decide to try. … We liked the idea of putting the materials in the binder so that way if that particular CPA is not there that day those materials are already prepped so they can just be taken out of the binder and used. That might be something we're gonna try.

      Getting staff buy-in

      Managing implementation also involved getting buy-in from WIC staff, especially those who did not participate in performance improvement teams. As 1 staff member described: “You really need buy-in from the staff—just because they yes you to death doesn't mean they are totally into the project.” Breastfeeding promotion as a core clinic value facilitated buy-in at some clinics whereas others described challenges related to BF promotion as “not a priority.” Buy-in was also challenged by staff resources, time constraints, and perceptions of intervention as “extra work” especially related to extra YCDI documentation required by staff. In some instances, resistance from non-team staff created further challenges related to distribution of responsibility among all staff. Staff training was affected by buy-in and difficulty setting aside time for trainings because of staff turnover, vacations, and clinic rotations.
      Staff described several strategies designed to improve buy-in and facilitate training, including framing YCDI as a skill-building exercise and treating enrollment as a competition among nutritionists. Teams reiterated the importance of getting input from non-team staff regarding implementation planning. Staff also reported developing creative training protocols to promote buy-in that were sensitive to staff skill level and confidence with intervention components:In the staff training, we're gonna do some mock targeted group discussions. We're gonna go over setting up those second-trimester materials again and what happens in that individual session because some of our CPAs have enrolled, you know, 18 people and some have enrolled 1. So we found that the people who are using it and doing it are really comfortable because they're practicing. And the people who are, maybe, offsite most of the time but want to be involved don't get the opportunity to practice as often because they're not as comfortable. So we'll use that meeting time to practice.

      Facilitating proactive participant contact

      To offer early postpartum support, nutritionists needed to know when enrolled mothers had given birth. However, discovering mothers had given birth emerged as a challenge, in part because it required proactive contact with WIC mothers when many clinics' routine practices relied on mothers to initiate postpartum contact. Other challenges included nutritionists' perceptions of WIC mothers as not responsive or too busy to speak after delivery. However, clinic supervisors questioned this characterization of mothers. For example, one supervisor wondered, “Was it the mother or was it the CPA?” and suggested that sometimes nutritionists were not confident with calling or proactively reaching out to mothers.
      To find out when mothers had given birth, clinics adapted the intervention to use peer counselors instead of nutritionists. Peer counselors already tended to reach out proactively to mothers around expected delivery dates using a variety of communication methods such as social media and messaging applications, and asking mothers to text selfies with their newborns. In addition, peer counselors’ field setting combined with partnerships with birthing hospitals facilitated contact: “… peer counselors doing rounds in the hospital helped identify when a BAPT mom gave birth.”

      Promoting advanced preparation

      Advanced preparation of tailored materials was described as critical to effective counseling visits with mothers and as an essential strategy to manage clinic flow. As 1 staff member noted, being prepared allowed nutritionists to be “… nice and calm and present when they're [mothers] here.” However, preparing materials in advance was challenged mainly by staff time constraints.
      To support implementation, clinics developed implementation practices that provided time for advanced preparation. Some clinics used their scheduling systems to create personalized reminders:A text message is sent to the CPA [nutritionist] 2 days before the appointment to remind them so that they can prepare for the targeted counseling. … I'm the peer counselor coordinator, so I look at the calendar, I send them a text …
      Others supported implementation by setting aside specific time for nutritionists to prepare materials, such as: “30-min blocked off once per week for team members.”
      Clinics also developed cheat sheets that organized necessary handouts by visit type to streamline preparation for visits and make staff less stressed. One clinic worked to systematize a specific tailoring process to save time (this was then shared with others in the Learning Community):And we were supposed to write in the back of that some notes for them about what tabs and what subsections to check for them. … We found that that was time-consuming and often, you know, our penmanship is not great. So we thought, why not prepare something once that we can then glue inside and highlight.
      A final strategy included involving clerical staff in tracking and organization:Support staff, they, you know, get everything ready the night before. So if we know 1 of the enrollees is coming in, they pull the chart and they gather whatever relevant information as far as education we need for that visit.

      Managing attendance at group sessions

      Targeted group discussions were reported as being well received by both staff members and attending participants. However, low attendance at targeted group discussion sessions (Table 1) was a challenge. Many clinics reported issues with no-shows and staff noted that “Spanish speaking working moms [were] hardest to schedule for.” Administrative data from clinics revealed that many clinics reported difficulty getting turnout for general group sessions, even apart from YCDI. Low turnout was most commonly attributed to perceptions of mothers’ tardiness. Administrative data also showed that staff developed adaptive practices to improve attendance at general group sessions: for example, by conducting impromptu groups with participants waiting for their appointments. However, attendance at YCDI-targeted group discussions was a persistent challenge throughout the intervention period and many clinics described having to conduct a one-on-one session instead of a group discussion.

      Discussion

      The YCDI intervention provided an opportunity to qualitatively investigate the implementation of a multicomponent intervention to promote EBF in the WIC setting from the perspective of the WIC staff actively engaged in the implementation. This study describes implementation benefits perceived by multidisciplinary WIC staff and how implementation was shaped by the interplay between intervention components and WIC clinic settings such as clinic resources (eg, time constraints, staffing levels), routine practices, values, and perceptions of mothers enrolled in WIC. Themes highlighted how clinics achieved implementation with a range of local strategies developed by staff and informed by Learning Community peers to navigate challenges, build on facilitators, and promote implementation.
      Many of the implementation challenges and facilitators presented here were consistent with previous research on integrating new health initiatives within WIC clinics, as well as implementing baby-friendly initiatives in community health centers.
      • World Health Organization UNICEF
      Protecting, Promoting and Supporting Breast-feeding: The Special Role of Maternity Services.
      For example, several studies also found that organizational settings such as staffing resources related to shortages, turnover, and time constraints
      • Nickel N.C.
      • Taylor E.C.
      • Labbok M.H.
      • Weiner B.J.
      • Williamson N.E.
      Applying organization theory to understand barriers and facilitators to the implementation of baby-friendly: a multi-site qualitative study.
      • Schmied V.
      • Gribble K.
      • Sheehan A.
      • Taylor C.
      • Dykes F.C.
      Ten steps or climbing a mountain: a study of Australian health professionals’ perceptions of implementing the baby friendly health initiative to protect, promote and support breastfeeding.
      • Thomson G.
      • Bilson A.
      • Dykes F.
      Implementing the WHO/UNICEF Baby Friendly Initiative in the community: a ‘hearts and minds’ approach.
      • Wheat S.
      Breastfeeding initiative.
      • Vincent S.
      Oldham—the second PCT in the UK to gain the award.
      • Seth J.G.
      • Isbell M.G.
      • Atwood R.D.
      • Ray T.C.
      Addressing language barriers in client-centered health promotion.
      ; staff resistance to changing routine practices
      • Vincent S.
      Oldham—the second PCT in the UK to gain the award.
      • Semenic S.
      • Childerhose J.E.
      • Lauziere J.
      • Groleau D.
      Barriers, facilitators, and recommendations related to implementing the Baby-Friendly Initiative (BFI): an integrative review.
      ; and perceptions of the intervention as time-consuming
      • Schmied V.
      • Gribble K.
      • Sheehan A.
      • Taylor C.
      • Dykes F.C.
      Ten steps or climbing a mountain: a study of Australian health professionals’ perceptions of implementing the baby friendly health initiative to protect, promote and support breastfeeding.
      • Semenic S.
      • Childerhose J.E.
      • Lauziere J.
      • Groleau D.
      Barriers, facilitators, and recommendations related to implementing the Baby-Friendly Initiative (BFI): an integrative review.
      acted as challenges to implementation. Other studies also identified perceptions of mothers’ culture, preferences, and hectic lives as challenges
      • Nickel N.C.
      • Taylor E.C.
      • Labbok M.H.
      • Weiner B.J.
      • Williamson N.E.
      Applying organization theory to understand barriers and facilitators to the implementation of baby-friendly: a multi-site qualitative study.
      • Thomson G.
      • Bilson A.
      • Dykes F.
      Implementing the WHO/UNICEF Baby Friendly Initiative in the community: a ‘hearts and minds’ approach.
      • Semenic S.
      • Childerhose J.E.
      • Lauziere J.
      • Groleau D.
      Barriers, facilitators, and recommendations related to implementing the Baby-Friendly Initiative (BFI): an integrative review.
      particularly to attendance at group sessions.
      • Krummel D.
      • Semmens E.
      • MacBride A.M.
      • Fisher B.
      Lessons learned from the mothers’ overweight management study in 4 West Virginia WIC offices.
      In addition, studies similarly showed that implementation facilitators included involving staff from multiple levels within an organization, inviting staff to contribute to planning efforts, and promoting collaborative learning across health care facilities, such as through a Learning Community.
      • Semenic S.
      • Childerhose J.E.
      • Lauziere J.
      • Groleau D.
      Barriers, facilitators, and recommendations related to implementing the Baby-Friendly Initiative (BFI): an integrative review.
      Although this and similar studies described routine organizational practices as challenges,
      • Vincent S.
      Oldham—the second PCT in the UK to gain the award.
      • Semenic S.
      • Childerhose J.E.
      • Lauziere J.
      • Groleau D.
      Barriers, facilitators, and recommendations related to implementing the Baby-Friendly Initiative (BFI): an integrative review.
      this study is unique in that it also presents routine organizational practices as potential facilitators upon which staff were able to build. For example, staff were able to build on communication routines (eg, daily huddles) and contact routines (eg, peer counselors postpartum contact with mothers) to facilitate implementation.
      • Feldman M.S.
      • Pentland B.T.
      Reconceptualizing organizational routines as a source of flexibility and change.
      Additionally, whereas other studies described lack of BF knowledge or confidence among staff as a challenge,
      • Wheat S.
      Breastfeeding initiative.
      • Vincent S.
      Oldham—the second PCT in the UK to gain the award.
      this did not emerge as a challenge in the current analysis. Rather, discovering gaps in staff BF knowledge emerged as an implementation benefit because it provided an opportunity for participating clinics to identify staff development needs that could be addressed through technical and experiential support offered through Learning Community Sessions along with a variety of existing BF training opportunities in NYS WIC. Previous studies characterized both sharing implementation experiences across agencies and accessibility of training opportunities as implementation facilitators.
      • Semenic S.
      • Childerhose J.E.
      • Lauziere J.
      • Groleau D.
      Barriers, facilitators, and recommendations related to implementing the Baby-Friendly Initiative (BFI): an integrative review.
      Other emergent implementation benefits, including perceptions of improved BF knowledge and counseling confidence, were also consistent with previous research.
      • Khoury A.J.
      • Hinton A.
      • Mitra A.K.
      • Carothers C.
      • Foretich C.
      Improving breastfeeding knowledge, attitudes, and practices of WIC clinic staff.
      It is possible that perceived benefits or successes of implementing may have encouraged organizations to endure implementation challenges.
      • Weiner B.J.
      • Lewis M.A.
      • Linnan L.A.
      Using organization theory to understand the determinants of effective implementation of worksite health promotion programs.
      This study extends previous research on implementation challenges and facilitators in clinic settings by highlighting specific staff-identified core implementation processes. This is important because challenges, facilitators, and strategies can be specific to both implementation processes and clinic settings
      • Weiner B.J.
      • Lewis M.A.
      • Linnan L.A.
      Using organization theory to understand the determinants of effective implementation of worksite health promotion programs.
      : 1 clinic may struggle with attendance at group sessions but excel at supporting and delivering tailored counseling. In addition, many of the processes that emerged as critical or challenging were not anticipated a priori (eg, proactive contact by nutritionists) and may be instrumental considerations for similar implementations in other WIC clinics. These findings underscore the benefits of using an approach that facilitates identification and understanding of important and emergent implementation processes, such as the ongoing qualitative data collection used in this study.
      Similarly, a contribution of this study is the description of strategies developed by clinics to facilitate implementation. Several implementation strategies described here were consistent with previous studies including creating communication plans, making adjustments to employee work schedules to support activities, developing site-specific training protocols, and expanding intervention involvement to include clerical staff.
      • Weiner B.J.
      • Lewis M.A.
      • Linnan L.A.
      Using organization theory to understand the determinants of effective implementation of worksite health promotion programs.
      • Meyers D.C.
      • Durlak J.A.
      • Wandersman A.
      The Quality Implementation Framework: a synthesis of critical steps in the implementation process.
      However, fewer studies described specific adaptations to the study protocol, including how site-specific settings shape those adaptations. Findings presented here complement research on the implementation of health programs within diverse organizations suggesting that adapting both organization-level practices and intervention components to achieve compatibility can facilitate implementation.
      • Weiner B.J.
      • Lewis M.A.
      • Linnan L.A.
      Using organization theory to understand the determinants of effective implementation of worksite health promotion programs.
      • Meyers D.C.
      • Durlak J.A.
      • Wandersman A.
      The Quality Implementation Framework: a synthesis of critical steps in the implementation process.
      • Forehand R.
      • Dorsey S.
      • Jones D.J.
      • Long N.
      • McMahon R.J.
      Adherence and flexibility: they can (and do) coexist!.
      These findings underline the need to identify core intervention components or active ingredients that should not be modified
      • McCleary N.
      • Duncan E.M.
      • Stewart F.
      • Francis J.J.
      Active ingredients are reported more often for pharmacologic than non-pharmacologic interventions: an illustrative review of reporting practices in titles and abstracts.
      and a set of allowable adaptations
      • Forehand R.
      • Dorsey S.
      • Jones D.J.
      • Long N.
      • McMahon R.J.
      Adherence and flexibility: they can (and do) coexist!.
      to maintain intervention efficacy in future work.
      Findings in this study suggest that implementation feasibility was improved, in part, through ongoing support from trainers and peers in the Learning Community. The WIC staff reported sharing issues and problem-solving strategies among one another, and findings suggested that many strategies would not have been adopted without recommendation by Learning Community peers (eg, advance preparation protocols). Some data presented here also suggested that a major role of peer learning was helping staff to reframe challenges in addressable terms, as discussed in Bartick et al.
      • Bartick M.
      • Edwards R.A.
      • Walker M.
      • Jenkins L.
      The Massachusetts Baby-Friendly Collaborative: lessons learned from an innovation to foster implementation of best practices.
      Implementation literature also points to the importance of ongoing technical assistance from intervention staff,
      • Meyers D.C.
      • Durlak J.A.
      • Wandersman A.
      The Quality Implementation Framework: a synthesis of critical steps in the implementation process.
      and in this study, intervention trainers were able to respond quickly to implementation concerns, misconceptions, and needs for assistance to support implementation as part of multiple Learning Community Sessions.
      Although several challenges were addressed through developing implementation strategies and support from the Learning Community, other challenges persisted throughout the intervention, such as low attendance at group discussions. It is possible that less modifiable factors such as mothers’ need for flexibility to respond to transportation and child care challenges might present implementation issues that are harder to address strategically.
      • Krummel D.
      • Semmens E.
      • MacBride A.M.
      • Fisher B.
      Lessons learned from the mothers’ overweight management study in 4 West Virginia WIC offices.
      These findings were limited by the small and nonrandom sample of participating WIC clinics whose perspectives may not represent clinics from across NYS. Also, data were gathered only from the performance improvement team, whose experiences may not be representative of staff across each WIC clinic and whose responses may be positively biased. In addition, data were collected as part of program monitoring and not evaluation research, restricting the questions and format of data collected and thus limiting insight into implementation feasibility by clinic characteristics. However, this analysis was strengthened by the triangulation of data from multiple formats (status reports, group discussion, and administrative data) over several time points, which enhanced the ability of findings to be confirmed by providing a more comprehensive perspective and affording staff the opportunity to review and extend descriptions of implementation experiences. Finally, although this analysis explored organizational-level factors influencing implementation feasibility, considerations of broader social and political landscapes (eg, peer counselor funding) and potential negative impacts of implementation were not explored.

      Implications for Research and Practice

      This research is unique in reporting the process of implementing a WIC-based initiative to promote EBF from the perspective of WIC staff implementers. Findings from this study show that successful intervention implementation by WIC clinics is likely to be shaped by compatibility between the clinic setting and core intervention components, although clinics can make some internal adjustments to modifiable factors to facilitate integration into WIC settings. These findings help to further understanding about implementing BF promotion initiatives in WIC care settings. Findings also highlight the need to consider fit with local clinic setting through assessments of organizational needs, routines, and less modifiable factors, for example, and to define allowable adaptations when they are implemented in WIC settings. Future work should collect clinic-specific setting information (eg, tenure of staff implementers) to describe and compare characteristics of clinics that were able to implement successfully and should assess factors related to feasibility of sustained implementation.

      Acknowledgments

      The authors thank the NYS WIC program, 12 participating local WIC clinics, Cicatelli Associates, Inc., Lynne Bortree, and Tricia Cassi of Vermont WIC for their support and participation in the successful implementation of this intervention.

      Conflict of Interest

      The authors have not stated any conflicts of interest.

      References

        • Ip S.
        • Chung M.
        • Raman G.
        • et al.
        Breastfeeding and maternal and infant health outcomes in developed countries.
        Evid Rep Technol Assess (Full Rep). 2007; 153: 1-186
        • Eidelman A.I.
        • Schanler R.J.
        • Johnston M.
        • et al.
        Breastfeeding and the use of human milk.
        Pediatrics. 2012; 129: e827-e841
      1. Rasmussen K.M. Latulippe M.E. Yaktine A. Review of WIC Food Packages: Proposed Framework for Revisions: Interim Report. National Academies Press, Washington, DC2016
      2. National Immunization Survey, Centers for Disease Control and Prevention. Rates of any and exclusive breastfeeding by socio-demographics among children born in 2013. https://www.cdc.gov/breastfeeding/data/nis_data/rates-any-exclusive-bf-socio-dem-2013.htm. Accessed January 11, 2017.

        • Renfrew M.J.
        • McCormick F.M.
        • Wade A.
        • Quinn B.
        • Dowswell T.
        Support for healthy breastfeeding mothers with healthy term babies.
        Cochrane Database Syst Rev. 2012; 5: CD001141
        • Patnode C.D.
        • Henninger M.L.
        • Senger C.A.
        • Perdue L.A.
        • Whitlock E.P.
        Primary care interventions to support breastfeeding: Updated evidence report and systematic review for the US Preventive Services Task Force.
        JAMA. 2016; 316: 1694-1705
        • Chung M.
        • Raman G.
        • Trikalinos T.
        • Lau J.
        • Ip S.
        Interventions in primary care to promote breastfeeding: an evidence review for the U.S. Preventive Services Task Force.
        Ann Intern Med. 2008; 149: 565-582
        • Reeder J.A.
        • Joyce T.
        • Sibley K.
        • Arnold D.
        • Altindag O.
        Telephone peer counseling of breastfeeding among WIC participants: a randomized controlled trial.
        Pediatrics. 2014; 134: e700-e709
        • Hildebrand D.A.
        • McCarthy P.
        • Tipton D.
        • Merriman C.
        • Schrank M.
        • Newport M.
        Innovative use of influential prenatal counseling may improve breastfeeding initiation rates among WIC participants.
        J Nutr Educ Behav. 2014; 46: 458-466
        • Weiner B.J.
        • Lewis M.A.
        • Linnan L.A.
        Using organization theory to understand the determinants of effective implementation of worksite health promotion programs.
        Health Educ Res. 2009; 24: 292-305
        • Nickel N.C.
        • Taylor E.C.
        • Labbok M.H.
        • Weiner B.J.
        • Williamson N.E.
        Applying organization theory to understand barriers and facilitators to the implementation of baby-friendly: a multi-site qualitative study.
        Midwifery. 2013; 29: 956-964
        • Isbell M.G.
        • Seth J.G.
        • Atwood R.D.
        • Ray T.C.
        A client-centered nutrition education model: lessons learned from Texas WIC.
        J Nutr Educ Behav. 2014; 46: 54-61
        • Edmunds L.S.
        • Lee F.F.
        • Eldridge J.D.
        • Sekhobo J.P.
        Outcome evaluation of the You Can Do It initiative to promote exclusive breastfeeding among women enrolled in the New York State WIC program by race/ethnicity.
        J Nutr Educ Behav. 2017; 42: S162-S168
        • Bortree L.
        • Decher L.
        • Flynn K.
        Supporting Long Term Breastfeeding With the New WIC Food Packages—You Can Do It / WIC Can Help.
        Vermont Agency of Human Services, Burlington, VT2013
        • Gill S.L.
        • Reifsnider E.
        • Lucke J.F.
        • Mann A.R.
        Predicting breast-feeding attrition: adapting the Breast-feeding Attrition Prediction Tool.
        J Perinat Neonatal Nurs. 2007; 21: 216-224
        • Janke J.R.
        Development of the Breast-Feeding Attrition Prediction Tool.
        Nurs Res. 1994; 43: 100-104
        • Hawkins R.P.
        • Kreuter M.
        • Resnicow K.
        • Fishbein M.
        • Dijkstra A.
        Understanding tailoring in communicating about health.
        Health Educ Res. 2008; 23: 454-466
        • Institute for Healthcare Improvement
        The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement.
        Institute for Healthcare Improvement, Boston, MA2003
        • Hsieh H.F.
        • Shannon S.E.
        Three approaches to qualitative content analysis.
        Qual Health Res. 2005; 15: 1277-1288
        • Miles M.B.
        • Huberman A.M.
        Qualitative Data Analysis: An Expanded Sourcebook.
        2nd ed. Sage, Thousand Oaks1994
        • World Health Organization UNICEF
        Protecting, Promoting and Supporting Breast-feeding: The Special Role of Maternity Services.
        World Health Organization, Geneva, Switzerland1989
        • Schmied V.
        • Gribble K.
        • Sheehan A.
        • Taylor C.
        • Dykes F.C.
        Ten steps or climbing a mountain: a study of Australian health professionals’ perceptions of implementing the baby friendly health initiative to protect, promote and support breastfeeding.
        BMC Health Serv Res. 2011; 11: 208
        • Thomson G.
        • Bilson A.
        • Dykes F.
        Implementing the WHO/UNICEF Baby Friendly Initiative in the community: a ‘hearts and minds’ approach.
        Midwifery. 2012; 28: 258-264
        • Wheat S.
        Breastfeeding initiative.
        Community Pract. 2001; 74: 90-91
        • Vincent S.
        Oldham—the second PCT in the UK to gain the award.
        Pract Midwife. 2011; 14: 33-34
        • Seth J.G.
        • Isbell M.G.
        • Atwood R.D.
        • Ray T.C.
        Addressing language barriers in client-centered health promotion.
        Health Promot Pract. 2015; 16: 320-328
        • Semenic S.
        • Childerhose J.E.
        • Lauziere J.
        • Groleau D.
        Barriers, facilitators, and recommendations related to implementing the Baby-Friendly Initiative (BFI): an integrative review.
        J Hum Lact. 2012; 28: 317-334
        • Krummel D.
        • Semmens E.
        • MacBride A.M.
        • Fisher B.
        Lessons learned from the mothers’ overweight management study in 4 West Virginia WIC offices.
        J Nutr Educ Behav. 2010; 42: S52-S58
        • Feldman M.S.
        • Pentland B.T.
        Reconceptualizing organizational routines as a source of flexibility and change.
        Adm Sci Q. 2003; 48: 94-118
        • Khoury A.J.
        • Hinton A.
        • Mitra A.K.
        • Carothers C.
        • Foretich C.
        Improving breastfeeding knowledge, attitudes, and practices of WIC clinic staff.
        Public Health Rep. 2002; 117: 453-462
        • Meyers D.C.
        • Durlak J.A.
        • Wandersman A.
        The Quality Implementation Framework: a synthesis of critical steps in the implementation process.
        Am J Community Psychol. 2012; 50: 462-480
        • Forehand R.
        • Dorsey S.
        • Jones D.J.
        • Long N.
        • McMahon R.J.
        Adherence and flexibility: they can (and do) coexist!.
        Clin Psychol Sci Prac. 2010; 17: 258-264
        • McCleary N.
        • Duncan E.M.
        • Stewart F.
        • Francis J.J.
        Active ingredients are reported more often for pharmacologic than non-pharmacologic interventions: an illustrative review of reporting practices in titles and abstracts.
        Trials. 2013; 14: 146
        • Bartick M.
        • Edwards R.A.
        • Walker M.
        • Jenkins L.
        The Massachusetts Baby-Friendly Collaborative: lessons learned from an innovation to foster implementation of best practices.
        J Hum Lact. 2010; 26: 405-411
      3. US Department of Agriculture, Economic Research Service. Rural-urban commuting area codes. https://www.ers.usda.gov/data-products/rural-urban-commuting-area-codes/documentation/. Accessed May 12, 2017.