Research Brief| Volume 48, ISSUE 10, P723-729.e1, November 2016

Download started.


Development and Testing of the Toronto Nutrition Environment Measures Survey–Store (ToNEMS-S)

  • Author Footnotes
    † Mr Lo was affiliated with Toronto Food Strategy, Toronto Public Health, Toronto, Ontario, Canada at the time this study was completed.
    Brian K. Lo
    Address for correspondence: Brian K. Lo, MPH, RD, Division of Nutritional Sciences, 417 Savage Hall, Cornell University, Ithaca, NY 14853; Phone: (647) 290-0522
    † Mr Lo was affiliated with Toronto Food Strategy, Toronto Public Health, Toronto, Ontario, Canada at the time this study was completed.
    Division of Nutritional Sciences, Cornell University, Ithaca, NY
    Search for articles by this author
  • Leia M. Minaker
    Propel Centre for Population Health Impact, University of Waterloo, Waterloo, Ontario, Canada
    Search for articles by this author
  • Catherine L. Mah
    Division of Community Health and Humanities, Faculty of Medicine, Memorial University, St. John's, Newfoundland and Labrador, Canada

    Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
    Search for articles by this author
  • Brian Cook
    Toronto Food Strategy, Toronto Public Health, Toronto, Ontario, Canada
    Search for articles by this author
  • Author Footnotes
    † Mr Lo was affiliated with Toronto Food Strategy, Toronto Public Health, Toronto, Ontario, Canada at the time this study was completed.
Published:August 27, 2016DOI:



      To describe the adaptation and inter-rater reliability assessment of the Toronto Nutrition Environment Measures Survey–Store.


      A total of 55 food stores (2 bakeries, 2 dollar stores, 19 gas/convenience stores, 18 grocery stores, 4 pharmacies, 8 specialty stores, and 2 superstores) were assessed in 2014. Cohen's kappa and intra-class correlation coefficients (ICC) were assessed for inter-rater reliability for product availability, price, quality, ethno-cultural accessibility scores, and overall store scores.


      Inter-rater reliability was high for availability (average kappa = 0.91), price (average ICC = 0.964), ethno-cultural accessibility score (ICC = 0.981), and overall store scores (ICC = 0.991).

      Conclusions and Implications

      Measures in the Toronto Nutrition Environment Measures Survey–Store demonstrated excellent inter-rater agreement. Limited access to fruits and vegetables and ethno-cultural foods are related to diet-related chronic diseases, particularly among immigrants. Findings of the study may be useful in guiding public health programming.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access

      SNEB Member Login

      SNEB Members, full access to the journal is a member benefit. Login via the SNEB Website to access all journal content and features.


      Subscribe to Journal of Nutrition Education and Behavior
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Egger G.
        • Swinburn B.
        An “ecological” approach to the obesity pandemic.
        BMJ. 1997; 315: 477-480
        • Swinburn B.
        • Kraak V.
        • Rutter H.
        • et al.
        Strengthening of accountability systems to create healthy food environments and reduce global obesity.
        Lancet. 2015; 385: 2534-2545
        • Ni Mhurchu C.
        • Vandevijvere S.
        • Waterlander W.
        • et al.
        Monitoring the availability of healthy and unhealthy foods and non-alcoholic beverages in community and consumer retail food environments globally.
        Obes Rev. 2013; 14: 108-119
        • Caspi C.E.
        • Sorensen G.
        • Subramanian S.V.
        • Kawachi I.
        The local food environment and diet: a systematic review.
        Health Place. 2012; 18: 1172-1187
      1. Applied Research Program, Division of Cancer Central and Population Sciences, National Cancer Institute. Measures of the Food Environment. Accessed August 19, 2016.

        • Story M.
        • Kaphingst K.M.
        • Robinson-O'Brien R.
        • Glanz K.
        Creating healthy food and eating environments: policy and environmental approaches.
        Annu Rev Public Health. 2008; 29: 253-272
        • Glanz K.
        • Sallis J.F.
        • Saelens B.E.
        • Frank L.D.
        Nutrition Environment Measures Survey in Stores (NEMS-S): development and evaluation.
        Am J Prev Med. 2007; 32: 282-289
        • Saelens B.E.
        • Glanz K.
        • Sallis J.F.
        • Frank L.D.
        Nutrition Environment Measures Study in restaurants (NEMS-R): development and evaluation.
        Am J Prev Med. 2007; 32: 273-281
        • Byker Shanks C.
        • Jilcott Pitts S.
        • Gustafson A.
        Development and validation of a farmers' market audit tool in rural and urban communities.
        Health Promot Pract. 2015; 16: 859-866
        • Lo B.K.C.
        • Minaker L.
        • Chan A.N.T.
        • Hrgetic J.
        • Mah C.L.
        Adaptation and validation of a nutrition environment measures survey for university grab-and-go establishments.
        Can J Diet Pract Res. 2016; 77: 17-24
        • Voss C.
        • Klein S.
        • Glanz K.
        • Clawson M.
        Nutrition environment measures survey-vending: development, dissemination, and reliability.
        Health Promot Pract. 2012; 13: 425-430
        • Gloria C.T.
        • Steinhardt M.A.
        Texas nutrition environment assessment of retail food stores (TxNEA-S): development and evaluation.
        Public Health Nutr. 2010; 13: 1764-1772
        • Martins P.A.
        • Cremm E.C.
        • Leite F.H.M.
        • Maron L.R.
        • Scagliusi F.B.
        • Oliveira M.A.
        Validation of an adapted version of the Nutrition Environment Measurement Tool for Stores (NEMS-S) in an urban area of Brazil.
        J Nutr Educ Behav. 2013; 45: 785-792
        • Glanz K.
        • Sallis J.F.
        • Saelens B.E.
        • Frank L.D.
        Healthy nutrition environments: concepts and measures.
        Am J Health Promot. 2005; 19 (ii): 330-333
        • Murphy S.P.
        • Barr S.I.
        Food guides reflect similarities and differences in dietary guidance in three countries (Japan, Canada, and the United States).
        Nutr Rev. 2007; 65: 141-148
        • Moore S.
        • Lloyd B.
        Improving the comparability of national estimates of fruit and vegetable consumption for cross-national studies of dietary patterns.
        Food Nutr Bull. 2012; 33: 312-317
        • Ball K.
        • Lamb K.E.
        • Costa C.
        • et al.
        Neighbourhood socioeconomic disadvantage and fruit and vegetable consumption: a seven countries comparison.
        Int J Behav Nutr Phys Act. 2015; 12: 68
      2. Library and Archives Canada. Ethno-cultural groups. Accessed April 1, 2016.

        • Stern D.
        • Poti J.M.
        • Ng S.W.
        • Robinson W.R.
        • Gordon-Larsen P.
        • Popkin B.M.
        Where people shop is not associated with the nutrient quality of packaged foods for any racial-ethnic group in the United States.
        Am J Clin Nutr. 2016; 103: 1125-1134
        • Conger K.J.
        • Reeb B.T.
        • Chan S.Y.S.
        Racial–ethnic minority youth in rural America: theoretical perspectives, conceptual challenges, and future directions.
        in: Crockett L.J. Carlo G. Racial Ethnic Minority Youth and Families in the United States: Theory, Research, and Applications. Springer International Publishing, Basel, Switzerland2016: 13-36
      3. Statistics Canada. Immigration and ethnocultural diversity in Canada. Accessed December 13, 2015.

        • Delisle H.
        Findings on dietary patterns in different groups of African origin undergoing nutrition transition.
        Appl Physiol Nutr Metab. 2010; 35: 224-228
        • Désilets M.-C.
        • Rivard M.
        • Shatenstein B.
        • Delisle H.
        Dietary transition stages based on eating patterns and diet quality among Haitians of Montreal, Canada.
        Public Health Nutr. 2007; 10: 454-463
        • Rose D.
        • Bodor J.N.
        • Hutchinson P.L.
        • Swalm C.M.
        The importance of a multi-dimensional approach for studying the links between food access and consumption.
        J Nutr. 2010; 140: 1170-1174
      4. City of Toronto. Kingston-Galloway neighbourhood improvement area snapshot. Social Policy, Analysis, and Research Social Development, Finance, and Administration, Toronto, Canada.,%20Finance%20&%20Administration/Shared%20Content/Demographics/PDFs/NIA_2011_Profiles/2011-NIA-KingstonGalloway.pdf. Accessed March 20, 2016.

      5. Statistics Canada. 2011 National household survey profile—Federal electoral district (2013 Representation Order). Accessed April 1, 2016.

      6. Health Canada. Eating well with Canada's food guide. Accessed August 19, 2016.

      7. Buxton, W, Sniderman, R. Iteration in the design of the human-computer interface. Proceedings of the 13th annual meeting. Rexdale, Ontario: Human Factors Association of Canada; 1980:72–81.

        • Tomlinson M.
        • Solomon W.
        • Singh Y.
        • et al.
        The use of mobile phones as a data collection tool: a report from a household survey in South Africa.
        BMC Med Inform Decis Mak. 2009; 9: 51
        • Bouwman H.
        • de Reuver M.
        • Heerschap N.
        • Verkasalo H.
        Opportunities and problems with automated data collection via smartphones.
        Mob Media Commun. 2013; 1: 63-68
        • Shafique M.A.
        • Hato E.
        Travel mode detection with varying smartphone data collection frequencies.
        Sensors. 2016; 16: 1-24
        • Rundle A.G.
        • Sheehan D.M.
        • Quinn J.W.
        • et al.
        Using GPS data to study neighborhood walkability and physical activity.
        Am J Prev Med. 2016; 50: e65-e72
        • Lovasi G.S.
        • Grady S.
        • Rundle A.
        Steps forward: review and recommendations for research on walkability, physical activity and cardiovascular health.
        Public Health Rev. 2012; 33: 484-506
        • Frank L.D.
        • Saelens B.E.
        • Powell K.E.
        • Chapman J.E.
        Stepping towards causation: do built environments or neighborhood and travel preferences explain physical activity, driving, and obesity?.
        Soc Sci Med. 2007; 65: 1898-1914
        • Hosler A.S.
        • Dharssi A.
        Reliability of a survey tool for measuring consumer nutrition environment in urban food stores.
        J Public Health Manag Pract. 2011; 17: E1-E8
        • Rimkus L.
        • Powell L.M.
        • Zenk S.N.
        • et al.
        Development and reliability testing of a food store observation form.
        J Nutr Educ Behav. 2013; 45: 540-548
        • Minaker L.M.
        • Raine K.D.
        • Wild T.C.
        • Nykiforuk C.I.J.
        • Thompson M.E.
        • Frank L.D.
        Objective food environments and health outcomes.
        Am J Prev Med. 2013; 45: 289-296
        • Farley T.A.
        • Rice J.
        • Bodor J.N.
        • Cohen D.A.
        • Bluthenthal R.N.
        • Rose D.
        Measuring the food environment: shelf space of fruits, vegetables, and snack foods in stores.
        J Urban Health Bull N Y Acad Med. 2009; 86: 672-682
        • John J.A.S.
        • Johnson C.M.
        • Sharkey J.R.
        • Dean W.R.
        • Arandia G.
        Empowerment of promotoras as promotora–researchers in the Comidas Saludables & Gente Sana en las Colonias del Sur de Tejas (Healthy Food and Healthy People in South Texas Colonias) Program.
        J Prim Prev. 2013; 34: 41-57
      8. Center for Health Behavior Research, University of Pennsylvania. NEMS workshops. Accessed April 1, 2016.

        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Müller R.
        • Büttner P.
        A critical discussion of intraclass correlation coefficients.
        Stat Med. 1994; 13: 2465-2476
        • Shrout P.E.
        • Fleiss J.L.
        Intraclass correlations: uses in assessing rater reliability.
        Psychol Bull. 1979; 86: 420-428
        • Cicchetti D.V.
        Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology.
        Psychol Assess. 1994; 6: 284-290
        • Laroche M.
        • Kim C.
        • Tomiuk M.A.
        • Bélisle D.
        Similarities in Italian and Greek multidimensional ethnic identity: some implications for food consumption.
        Can J Adm Sci Rev Can Sci Adm. 2005; 22: 143-167
        • Holmboe-Ottesen G.
        • Wandel M.
        Changes in dietary habits after migration and consequences for health: a focus on South Asians in Europe.
        Food Nutr Res. 2012; 56: 1-13
        • Astrup A.
        • Dyerberg J.
        • Selleck M.
        • Stender S.
        Nutrition transition and its relationship to the development of obesity and related chronic diseases.
        Obes Rev. 2008; 9: 48-52
        • Van Rompay M.I.
        • McKeown N.M.
        • Castaneda-Sceppa C.
        • Falcon L.M.
        • Ordovas J.M.
        • Tucker K.L.
        Acculturation and sociocultural influences on dietary intake and health status among Puerto Rican adults in Massachusetts.
        J Acad Nutr Diet. 2012; 112: 64-74
        • McDonald J.T.
        • Kennedy S.
        Is migration to Canada associated with unhealthy weight gain? Overweight and obesity among Canada's immigrants.
        Soc Sci Med. 2005; 61: 2469-2481
        • Hadley C.
        • Patil C.L.
        • Nahayo D.
        Difficulty in the food environment and the experience of food insecurity among refugees resettled in the United States.
        Ecol Food Nutr. 2010; 49: 390-407
        • Patil C.L.
        • Hadley C.
        • Nahayo P.D.
        Unpacking dietary acculturation among new Americans: results from formative research with African refugees.
        J Immigr Minor Health Cent Minor Public Health. 2009; 11: 342-358
        • Zenk S.N.
        • Odoms-Young A.
        • Dallas C.
        • et al.
        “You have to hunt for the fruits, the vegetables”: environmental barriers and adaptive strategies to acquire food in a low-income African-American neighborhood.
        Health Educ Behav. 2011; 38: 282-292