Home Eating Seasonally and Locally Growing Food Security in NY Communities Usability of the Northeast Regional Food Guide
Usability of the Northeast Regional Food Guide
Food guides provide a bridge between the science of nutrition and the general public, yet they are rarely tested for usability. The Northeast Regional Food Guide (NRFG) was developed to help Northeastern consumers follow healthful diets while emphasizing foods that are grown and processed locally. The purpose of the study was to evaluate the usability of the NRFG. Using a randomized pretest-posttest design with 27 mothers in the Special Supplemental Nutrition Program for Women, Infants and Children (WIC), four substitution and comparison tasks were used to assess consumers' ability to apply both the Dietary Guidelines and regional and seasonal eating recommendations. The challenges of seasonal research and the value of the tasks are discussed.
Food guides provide a bridge between the science of nutrition and the general public. Food guides "translate dietary standards and recommendations into simple and reliable nutrition education tools useful to consumers in improving the nutritive balance of dietary patterns." 1 To be effective, a food guide needs to be easily understood, memorable, and appropriate for the intended audience(s). Indeed, the strength of a food guide in encouraging healthful food choices lies in its usability.
Development, Testing, and Usability of Food Guides. Since the early 1900s, the U.S. Department of Agriculture (USDA) has developed several food guides for the American public. The first widely used food guide was Food for Fitness-A Daily Food Guide,2 commonly known as the Four Food Groups, which was developed in the mid-1950s. This guide was revised several times and used for many years, though one of its shortcomings was the failure to be an effective and efficient communications tool. 1 As Light and Cronin have noted nearly twenty years ago, not only does the Four Food Groups remain untested, but "there is no record in the nutrition literature of scientifically designed studies to test the usability aspects of any food guide."1
Following the Four Food Groups, the USDA's next major nutrition education tool was the Food Guide Pyramid,3 which is the implementation strategy for the U.S. Dietary Guidelines for Americans.4 Two of the philosophical goals for the guide were to be useful to the target audience and to be practical. 5 During the development of the Food Guide Pyramid, a market research firm did extensive research on the graphic design - testing figures such as a circle, blocks in a row, blocks in a circle, inverted pyramid, and pyramid, and later, bowls, shopping carts, and plates - to ensure that the guide communicated the desired messages of variety, proportionality, and moderation. 5 The graphics and text were "extensively peer reviewed by nutrition educators in the Federal government, Extension Service, and academia." 5 Other research was conducted to verify that the graphic did not convey misinformation to groups at increased nutritional risk - children, consumers with less than a high school education, and low-income consumers.5 Most of USDA's research on food guides has focused around how the food guides translate dietary recommendations and communicate them to consumers; research was not done to explore how consumers apply food guide information.
Some investigators have attempted to examine the effectiveness of their own guides. In 1983, Lynde et al 6 created a food guide for female Italian immigrants in Canada, and using a pre-post test questionnaire design they measured changes in nutrition knowledge and health beliefs. A food frequency questionnaire was employed to assess changes in dietary patterns. The researchers concluded that the guide increased knowledge in areas of nutrition of particular concern, including the use of food labels, causes and prevention of weight gain, and dental health. The researchers also found statistically significant improvement in responses to all questions regarding the healthfulness of cariogenic foods. Further, the frequency of consumption of four of the seven "marker" foods changed significantly in the direction that the guide promoted: torta (decrease), cheese (increase), eggs (increase), and carrots (increase).6 This testing showed that the guide facilitated improvement in knowledge, attitudes, and behavior, in that the women showed the desired changes in all three of these measures.
Other studies have focused on the perceived usability of food guides. Gillespie and Roderuck7 created a nutrient guide, which grouped foods according to nutrient composition. This guide was one component of a three-lesson correspondence nutrition education program for middle-class parents of young children. The lessons addressed nutrients and their role in the diet, snacking, food labels and the relationship of nutrition to diet, and diet selection and evaluation. A post-test questionnaire asked parents if they had read the guide, if they found it useful, how difficult it was, and how it changed the family's food behavior. Gillespie and Roderuck found that 52% of the fathers (n=33) and 75% of the mothers (n=79) perceived the guide to be "moderately useful" or "very useful," while 82% of the fathers and 86% of the mothers rated the difficulty as "very easy," "easy," or "average." 6 Subjects also reported some food behavior changes, such as eating a wider variety of foods, eating more vegetables and fruits, and regulating their children's snacks. Thus, the authors concluded that the guide "has the potential for being a useful teaching tool."7
Gust et al 8 studied consumer perceptions of Canada's Food Guide to Healthy Eating by using group interviews with adult female consumers. Using a discussion guide, four group interviews were conducted, with women of similar age (older or younger than 40 years old) and education level (completed college or not) grouped together. Participants responded favorably to the overall presentation and perceived understandability of the guide, and they said that they would use it as a reference but not for meal planning. They suggested sample recipes and menus as resources that would aid them in applying the guide. The authors concluded that while actual comprehension of Canada's Food Guide to Healthy Eating was not evaluated, "participants overwhelmingly indicated that they see the Guide as straightforward and easy to understand."8 As seen in this study, qualitative perceptions of usability can be useful during the development phase, but more quantitative research on the translation of messages to dietary planning is needed after the guide has been completed to determine if the goals of the guide are achieved.
Edwards9 conducted an extensive evaluation to analyze implementation and assess effectiveness of the American Red Cross Nutrition Course. This program is taught in six two-hour sessions, and it uses the Food Wheel as the framework for the course10,11. The Food Wheel, an early version of the USDA's Food Guide Pyramid3, was designed with the following criteria to maximize usability: to use food groups as the organizing principle; to reflect the diversity of foods available and the ways those foods are popularly used; to allow flexibility in selecting foods to meet nutritional objectives; and to allow families and other groups eating together to use the guide by choosing different amounts of foods from the same menu. 12 A field test of the nutrition course assessed changes in the participant's nutrition knowledge, beliefs, and behavior through pre- and post-questionnaires and a 10-week follow-up telephone survey. With pre-post results for a sample of 885 participants, researchers found significant change in the mean from baseline to post-test in each of the three areas of interest, while there were no significant changes for the control group. A follow-up survey 10 weeks after the course on over one hundred participants found that nutrition beliefs remained constant, that nutrition knowledge declined, and that nutrition behavior improved significantly.9 These results substantiate the effectiveness of the course.
Studies of the effectiveness of nutrition education programs have shown improvement in nutrition knowledge, but these types of evaluations do not indicate whether consumers can use this knowledge in making food choices and planning meals.13,14,15 Other studies find a change in behavior, but it may not be clear how this change came about.16 Usability testing looks at an intermediate step between increasing knowledge and changing behavior. It looks at how food guides can be used to change behavior.
In one of the only studies of usability per se, Hunt, Gatenby, and Rayner17 evaluated the ability of consumers to apply the concepts underlying Britain's National Food Guide to food choices. In this study, performance was defined as how well the information was understood and recalled. Subjects performed four types of tasks to test the effect of exposure to the guide on understanding and recall of the guide's concepts. The two main concepts are nutritional equivalence (that foods within groups are equivalent in nutritional content) and proportionality. Proper proportion, or balance, is reflected in the ratio of the areas of the segments representing the food groups in the guide.17
In the substitution task, subjects were shown a picture of a shopping basket with 13 food items and then four pictures of the same arrangement of foods each with one food item removed. They were told which food had been removed. They were then asked to choose one item from 10 additional foods to replace the missing food while maintaining balance and healthfulness within the shopping basket. The foods in the basket were selected in approximately the same ratio as suggested by the guide. Subjects scored a point for each appropriate replacement food they chose, an appropriate food being one from the same group.17
In the first comparison task, subjects were shown photographs of four meals and asked to rank the meals in order of 'healthiness' (not defined) and 'balance' (the selected foods reflect the proportion of area in the graphic devoted to each food group). For each of the nine sets of meals, subjects got one point for identifying the most healthy and balanced meal and one point for correctly identifying the least healthy and least balanced meal. Subjects ranked shopping baskets similarly in a second comparison task.17
In the same study, 17 two additional tasks involved sorting foods into groups or judging the appropriate proportions for the ingredients of a composite food. The researchers concluded that the tasks measured actual, as opposed to declared understanding and recall of the guide concepts because the tasks mimicked decisions consumers might make in real life. Usability is an important goal of food guides, and this study represents a key piece of research on the usability of food guides.
The Northeast Regional Food Guide. The Northeast Regional Food Guide (NRFG)18 was developed to promote healthful diets and to encourage consumption of foods produced and processed locally. The NRFG and the USDA Food Guide Pyramid3 are congruent in several ways. First, the NRFG is consistent with the Dietary Guidelines for Americans.4 It is like the USDA Food Guide Pyramid in that a pyramid graphic and the same food groups and number and sizes of servings are used. Because one of the primary goals of this food guide is to strengthen local agriculture and build community food systems, two additional dietary guidelines were developed to convey this interest. These are: "Choose a diet with plenty of foods produced in the Northeast" and "Eat a seasonally varied diet-especially with respect to fruits and vegetables." The NRFG includes a list of fruits and vegetables, by season, that are presently produced in the Northeastern United States and available either through the conventional market or through a direct marketing arrangement such as a Community Supported Agriculture farm (CSA) or a farmers' market. The poster is accompanied by a set of fact sheets that explains the justification for and approaches to emphasizing local foods in choosing diets.18
DESCRIPTION OF THE INTERVENTION
The effectiveness of the NRFG needed to be investigated. The study was designed to assess the ability of consumers to apply both the nutritional as well as the regional and seasonal eating recommendations that are suggested in the NRFG. The primary hypothesis of the study was that instruction about the NRFG would result in greater understanding of local and seasonal concepts than instruction about the USDA Food Guide Pyramid. To test this hypothesis, a pre-test post-test experimental design was used. Twenty-seven WIC clients were recruited to participate in the study.
In a pre-test questionnaire information was obtained about existing eating, cooking, and food shopping behavior; nutrition knowledge; attitudes about local foods; gardening; and demographics. All subjects received an individual nutrition education lesson on dietary guidance from the researcher. Control group subjects learned about the USDA Food Pyramid, while the experimental group subjects learned about the NRFG. Subjects in both groups completed four tasks to assess the usability of the food guide. The lesson and tasks took about 30 minutes for the control group and 45 minutes for the experimental group. The follow-up questionnaire was administered about two months after the intervention. This questionnaire contained questions from the pre-test designed to measure change in knowledge, attitudes, and behavior. The use of the tasks was the key to examining usability, so it is this aspect of the study that needs to be examined further.
DESCRIPTION OF THE EVALUATION
Tasks used in this study were based on the work of Hunt, Gatenby, and Rayner's study evaluating Britain's National Food Guide, 17 and they were adapted to test the appropriate concepts and to involve the customary foods in the northeastern United States. The four tasks were scored so that points were given for wrong answers, and a perfect score was 0.
In Task 1, "Ranking menus for healthfulness," subjects were given four written menus with serving sizes for an entire day (breakfast, lunch, and dinner) and asked to rank them from most healthy to least healthy, according to the Dietary Guidelines. The foods varied in nutrient content (most notably fat content) and preparation methods. This task was scored by taking the absolute value of the difference between the subject's response and the correct answer for each menu, with the mistakes on the most healthy and least healthy menus counting double because it was most important to be able to differentiate these menus from the others. Then the total score for the four menus was added up.
Task 2, "Local/seasonal 5 A Day" was developed, in which subjects were given a list of 10 fruits and vegetables and asked to choose their "5 A Day" (5 servings of fruits and vegetables) from local fruits and vegetables. They were also reminded that it was the fall season. In this regard, items such as fresh strawberries which can be grown locally but were not in season at the time of the study, were not considered an appropriate choice. Subjects were instructed to assume that all frozen, canned, or dried fruits and vegetables are from local sources if the item can be grown in the Northeast.
Subjects were given zero points for selecting a correct answer, four points for selecting a food that is not locally grown, and two points for a food that was locally grown but is not in season. Because some "summer" vegetables were still available in early fall, the subjects who completed this task in September had two additional foods on the list scored as correct answers. In the control group, this change affected the scoring of all 6 (100%) September surveys. In the experimental group, this only affected 3 of the 6 (50%) September surveys. In task 3, "Shopping basket substitution," subjects selected a replacement food for four different items in the shopping basket. The term "shopping basket" was used for a grouping of foods a person might purchase at the grocery store. The scoring gave subjects 0 points for selecting a low-fat food from the same food group, 1 point for a high-fat food from the same group, and 4 points for a food from the wrong food group.
In Task 4, "Seasonal shopping baskets," subjects were shown four shopping baskets with a selection of foods from all groups in the approximate ratio of the Food Guide Pyramid, and the produce was locally available in a certain season. Subjects were asked to identify the season to which each basket most closely corresponded. For each season there was one corresponding basket. Scoring was such that for every basket, subjects got zero points for the correct season, 1 point for the subsequent season, three points for the previous season, and four points for the remaining season. This scoring method was used because items that can be stored or that have a long harvest might be available into the subsequent season, and a few items might have an early harvest and be available in the previous season. After this section was completed, the correct answers were provided, and the subjects were then asked to select one additional food appropriate for each season's basket: one that is locally grown in the given season. Subjects chose from 10 items, and the list had at least two correct answers for each season. In scoring this section of the task, subjects got 0 points for a correct answer, 1 point for an out-of-season food, and 4 points for a food that can not be grown locally. The scores for the two sections were added together for the total score.
It has been observed that usability testing generally is not performed on food guides. Much research and effort goes into the development of guides, but this must be followed with field testing for effectiveness. Because of the unique nature of the NRFG, usability testing was clearly needed to see if the guide could simultaneously communicate nutrition and regional and seasonal messages. This study explored one strategy for usability testing on a regional food guide. A unique challenge of testing a regional food guide relates to seasonality. The study was planned such that the intervention would occur in the fall season, from mid-September to late October, and the follow-up would occur in the winter, from mid-November to late December.
The agricultural seasons are based on when frosts take place, so they are difficult to define and anticipate. In the 1998 season, many "summer" vegetables were still available through late September, which was confusing for some subjects when they were responding to questions about local and seasonal foods. Along the same lines, fruits and vegetables which are available in multiple seasons or can be stored for several seasons cause difficulty. Defining which foods can be grown in the Northeast region presents less of a challenge, though foods which can be grown in greenhouses or under other special conditions can add even more complexity to the issues of seasonality and locality. In order to detect seasonal changes in eating, the time period during which the research is conducted must be carefully considered. Administering the intervention over a shorter time period in the middle of a season would eliminate the variability in the seasonal availability between subjects. Perhaps the easiest times to do this type of research would be during the summer, when many fruits and vegetables are available, with follow-ups done in the middle of the winter, when little local produce is available fresh.
The tasks assessed whether consumers could apply nutrition messages to food shopping and menu planning. Testing such as this ensures that consumers have the ability to put knowledge to use in the way that nutritionists intend. Having subjects perform tasks instead of answering questions more closely represents situations they will encounter in daily life. It can also eliminate difficulties that may be encountered using subjects with low levels of literacy. Another potential benefit may be that this more active type of testing will hold the attention of subjects better than a lengthy questionnaire. These tasks are a valuable tool for usability testing, but their validity and reliability needs to be established. Future research should be done to further develop methods for usability testing.