Objective
Collecting ‘before’ and ‘after‘ ratings at the time of post-testing for program evaluation, reduces the likelihood of response-shift bias with a resulting increase in reliability. This method may be promising for program evaluation, especially where overestimation of initial behaviors are likely with a resulting improvement in intervention outcomes. Applying qualitative methods, the objective is to determine client preferences for retrospective approach and format, and implement those results in two tools of varying length.
Description
The Nutrition Education Evaluation Model (NEEM) was used to guide selection of approach and development of format for a non-traditional retrospective evaluation. Cognitive interviews were conducted at a food bank in 2006 and at Head Start and EFNEP sites, 2011-2018. Participants were low-income, English or bilingual Spanish speaking and available for an immediate interview. Several strategies were employed: concurrent and retrospective think alouds, paraphrasing probes, response format options and restating text. EFNEP educators offered opinions in unstructured discussions.
Evaluation
Study participants and EFNEP educators recommended one of four approaches: the Post then Pre. They preferred the format featuring each question on row along with two sets of response options flanking the centered question, use of a visual to portray each question, use of color to guide tool instructions, and presence of icons to support ‘before/then’ and ‘after/now’ concepts.
Conclusion and Implications
Two examples of the visual retrospective method were produced: Focus of Sweet Drinks…Now and Then for an English speaking low-income audience and Focus on Veggies…Now and Then for a Spanish speaking low-income audience. This qualitative method and the study results can be utilized by other researchers and practitioners.
Funding: 2015-68001-23280.
Appendix. Supplementary data
Article info
Identification
Copyright
© 2019 Published by Elsevier Inc.
