AgrAbility Virginia Program Evaluation Brief: 2023 Survey & Interview Results

Introduction‌

AgrAbility Virginia promotes safety, wellness, and accessibility on the farm through education, rehabilitative services, and assistive technology. AgrAbility Virginia is a partnership program between Virginia Tech, Virginia State University, Virginia Cooperative Extension, and Easterseals UCP. AgrAbility Virginia is funded by the United States Department of Agriculture (USDA) National Institute for Food and Agriculture (NIFA). As a statewide program, AgrAbility Virginia works closely with Virginia's rehabilitation and agricultural service delivery system to increase organizational capacity and provide the best quality education and services for farmers across Virginia. AgrAbility Virginia works in partnership with the Virginia Department of Aging and Rehabilitative Services (DARS) and others to make recommendations for farmers regarding assistive technologies and new farm procedures and protocols that will make farming more comfortable and safer.

To evaluate our programming, we used a utilization-focused summative-mixed methods evaluation to quantify and qualify any changes in quality of life, hopefulness, farm safety, and other factors that have taken place as a result of a farmer’s participation in AgrAbility Virginia programming. This brief shares results from two surveys—one for farmers (n=9) and one for service providers (n=5)—as well as farmer interviews (n=3). Both participating farmers and partner service providers expressed positive feedback about the program, with farmers noting positive changes in quality of life, accessibility and productivity and service providers describing AgrAbility as a valuable resource for their clientele. The findings from this evaluation will inform the work of AgrAbility Virginia during the 2023-2027 cycle of our USDA NIFA grant funding.

Farmers who participated in the survey and interview portions of our evaluation were overwhelmingly satisfied with their participation in the program, noting positive changes in their quality of life, as well as the accessibility and productivity of their farms. Though we had fewer responses from service providers (9.2% of those invited responded), service providers provided insight into the ways AgrAbility Virginia meets the needs of their farmer clients and the ways AgrAbility Virginia may expand engagement statewide.

Methods

The methods we used in this evaluation were based on the work of Kyle et al. (2017). Kyle et al. conducted a similar evaluation of Virginia’s AgrAbility program in 2016 using a convergent parallel mixed methods design following the work of Creswell and Plano Clark (2011). Mason and Niewolny also used these methods in an AgrAbility Virginia program evaluation published in 2021.

Using the convergent parallel mixed methods design explained by Creswell and Plano Clark, we collected and analyzed qualitative and quantitative data separately before engaging in joint analysis to identify commonalities across the two sets of data. For our 2023 program evaluation, we based our surveys on the farmer survey designed by Kyle et al. and adapted by Mason and Niewolny and the service provider survey created by Mason and Niewolny in 2021. These surveys comprised the quantitative portion of the evaluation. The farmer client survey included 30 questions, including Likert scale questions and open-ended questions. The service provider survey included 14 Likert scale and open-ended questions. For the qualitative (interview) portion of this study, we used the methodology designed by Kyle et al. These interviews were semi-structured and lasted 15-20 minutes.

On December 20, 2022, we sent a pre-recruitment email to farmers who have had significant participation in our program since 2019 (n=34). Two emails were returned as undeliverable. Additionally, one farmer had passed away and family members contacted our staff expressing interest in completing the survey on the farmer’s behalf. Not included on our contact list were six farmers that have had minimal contact with our staff (service entailed either a brief phone call or email exchange, or farmers had only begun working with AgrAbility Virginia staff in the two months before sending the survey). It was determined that these farmers did not have enough familiarity with our program’s offerings and/or interaction with program staff to participate in the evaluation. Emails with the link to the survey were sent on Monday, January 9th. The survey was sent by mail to one farmer who does not use the internet. Reminder emails were sent on Monday, January 23rd and Monday, January 30th. We conducted phone interviews with three farmers in February and March.

We contacted service providers who have interacted with our program since 2019 to invite them to complete a survey. Service providers who subscribe to our AgrAbility Virginia Network listserv (n=54) received the email via the listserv. Service providers were invited to participate via email on January 17th, January 23rd, and January 30th (we did not send a pre-recruitment email to service providers).

Farmer Survey Results

Of the 32 farmers who were successfully contacted, nine responded to our survey (28%) though some respondents chose not to provide data for some of the questions and prompts. The response rate for our 2021 program evaluation survey was 24%. For the 2021 survey, we mailed surveys to several farmers who had not provided an email address. In 2023, we sent only one survey by mail, all other were sent electronically. To enhance accessibility, we offered to complete the survey over the phone with anyone interested, but no farmers contacted us to request to participate over the phone. All responses to this program evaluation survey were electronic, no surveys were returned by mail.

Demographics

Farmer participants that completed this survey ranged in age from 43 to 75 years old. The mean age was 53. One participant was a farm family member/caregiver who did not provide their age. Three participants were military veterans, the farm family member/caregiver declined to answer. All eight respondents reported gender: two female and six male respondents (the farm family member answered this question on behalf of the farmer client). Those who listed their race self-identified as white (n=4), and Black or African American (n=4). These demographics roughly reflect those of our program participants as a whole. 53% of farmers served by AgrAbility Virginia in the 2019-2023 period represented historically underserved groups as defined by the USDA (beginning, socially disadvantaged, veteran, and/or limited resource farmers). Responses from this demographically- proportionate group are reflected below.

These farmers grow or raise a range of produce, crops, and livestock, such as fruits (n=2), vegetables (n=4), flowers (n=1), beef (n=2), poultry (n=2), feed grains, forages, and hay (n=1). One participant reported that they engage in agritourism and another reported producing Christmas trees or other trees/forest products. Participants farm on their own (n=2), have volunteers help on the farm (n=2), have immediate family who help (n=4), and one also hires full and part-time employees. The number of years that survey participants have been engaged in farming varied: with two reporting that they have been farming at least 20 years, one 9-15 years, one 6-9 years, two 1-3 years. All but one survey participants reported being rural farmers, with one listing their farm as urban.

Four respondents farm under two acres, two farm 2-5, one farms 101-200, and another listed their farm at over 200.

Survey participants are located in the following counties and cities: Appomattox (n=1), Bedford (n=1), Campbell (n=1), Chesterfield (n=1), Fluvanna (n=1), King and Queen (n=1), Richmond City (n=1). One respondent did not indicate their county.

AgrAbility Virginia also wanted to get a sense of where program participants sold their products, what kind of products they were selling, and how far away they needed to travel to sell their products. Two were not currently marketing products. Of the six that reported marketing products, two used farmers markets, two had an on-farm retail store or otherwise sold products on the farm with one of these participants listing agritourism as a market. Participants also listed wholesale (n=1), marketing co-op (n=1), commodity markets (n=1), livestock auction (n=2). Participants who sold products listed mileage to markets as 0-15 (n=1), 16-25 (n=1), and 26-50 (n=4).

Technical Assistance Recommendations and Assistive Technologies

From the survey, we learned that, as a result of their participation in the program, respondents are using a cart for a two-wheel tractor, long handled tools, an anti-fatigue mat for standing at market, a tool for turning irrigation on and off, seeders, garden carts, garden scooters and kneelers, and a wheeled bucket. AgrAbility also made the following suggestions to farmers: a four-wheeler and post-hole digger, and a tractor hoist. Respondents also noted that they have participated in seminars, used educational resources, contacted other agriculture service personnel. Most respondents reported that they “strongly agree” in response to the following prompts:

• “Information/education provided by AgrAbility Virginia has been useful to me and my farm” (n=7)

• “AgrAbility Virginia successfully assisted in finding necessary modified tools or equipment for my farm.” (n=6, one participant did not respond).

One farmer who noted that services were interrupted by the beginning of the COVID-19 pandemic, illness, and then losses in the family responded ‘neither agree nor disagree’ to both prompts.

Some participants were aware of additional resources available to them. In response to the prompt “I know of additional resources (including service providers) I can access regarding my health because of AgrAbility Virginia's Program,” participants responded that they strongly agree (n=1), agree (n=4), neither agree nor disagree (n=1), and somewhat disagree (n=1). Half of respondents did not list other programs they are receiving assistance from (n=4), but those who listed them included the Department of Aging and Rehabilitative Services, Virginia State University Small Farm Outreach Program, and a North Carolina A&T farming conference.

Quality of Life and Other Changes in Attitudes

Most clients indicated that AgrAbility was useful and made positive changes in their lives (Table 1). Six agreed (specifically, participants responded: ‘Somewhat Agree,’ n=1; ‘Agree,’ n=2; ‘Strongly Agree,’ n=3) that with AgrAbility Virginia’s assistance, they are more hopeful in their ability to meet their farming goals (while one answered ‘Neither Agree nor Disagree’) (Table 2). When considering quality of life improvements, the results were also positive (Table 3): four strongly agreed with the phrase “My quality of life has increased due to AgrAbility Virginia,” and one agreed, one somewhat agreed, and one responded ‘neither agree nor disagree.’ Overall, farmers feel secure in their ability to farm safely and productively (Table 4): three strongly agreed with the phrase “I am confident in my ability to continue farming safely and productively,” two selected ‘agree’ and two selected ‘somewhat agree.’ In responding to a question about changes in quality of life as a result of participating in AgrAbility Virginia programming (Table 5), one respondent noted “My quality of life has increased since working with AgrAbility. I no longer need to be on the ground and seeding. The seeder I was able to purchase will significantly increase my ability to plant (correctly spaced and depth) which will increase my yield in the next growing season. I also will be able to work more often as I will have decreased pain for working on the farm.” Another responded, “My quality of life has declined due to medical issues, but AgrAbility has helped me to stay positive about farming in the future.”

The group of farmers who responded to our survey were demographically representative of our overall population of farmers we serve. These farmers, from across Virginia, reported positive outcomes from their participation in the AgrAbility Virginia program including the use of assistive technologies suggested by AgrAbility staff, as well as the use of educational resources and networks provided by staff. Participants largely agreed that they experienced positive changes in their quality of life due to their participation in the AgrAbility Virginia program

Interview Findings

Four AgrAbility Virginia farmer clients expressed interest in participating in the interview portion of the evaluation. Interviews with three of these farmers occurred in February and March 2023. AgrAbility staff connected with the fourth interested farmer but determined them not to be eligible to participate based on their limited interaction with AgrAbility. Instead, AgrAbility Virginia noted the farmer’s interest in reopening a relationship with AgrAbility and reinitiated services during the call.

The three farmers who participated in the interview had overwhelmingly positive feedback to share about their experience working with AgrAbility staff and receiving support from the program. As we have found in earlier iterations of our program evaluation, some portion of this positive response could be due to self-selection, with those who had a positive experience in the program being more interested and willing to share their feedback. We also note that because the participants are asked to provide feedback directly to AgrAbility staff through these phone interviews, they may not feel comfortable providing constructive criticism or critique during this portion of the evaluation.

The farmers who participated in the interview portion of the evaluation are all small-scale farmers, two urban and one rural. Two are established farmers and gardeners who engage in vegetable production as a way to gain fulfillment during their retirement from other careers. The third is a beginning farmer with about five years of experience raising poultry and growing produce. All expressed gratitude to the staff and the program for helping them get assistive technologies that enable them to continue to farm.

Major themes that emerged from the interviews were farmers’ increased access to assistive technologies through their work with AgrAbility Virginia. All three participating farmers were able to take advantage of the Easterseals Bellows Fund, a small grant program for assistive technologies for people with disabilities. As AgrAbility Virginia farmer clients, these farmers received support in selecting tools that were adapted to fit their unique needs, and in filling out and submitting the application. The farmers express gratitude for this service in their answers (shown in Table 6).

Another theme all three farmer interviews included was feeling supported by AgrAbility staff. Of note was the accessibility of the staff at events and via email or phone communication, consistent follow-up, and quick response times. These responses are detailed in Table 6.

Farmers also noted changes to their quality of life, to the accessibility of their garden, and to their productivity. Discussions within these themes included ease of labor, increased energy, and reduced pain using adapted tools compared to conventional tools. Farmers also described the tools as boosting productivity, functionality, and as a ‘time-saver.’ Interviews with each of the three farmers included some discussion of at least one of these themes (quality of life, accessibility, and/or productivity). Quotes related to these themes are included in Table 6.

Service Provider Survey Results

We reached out to the AgrAbility Virginia Network (AVN) listserv to recruit participants for the service provider portion of our evaluation. There were 54 members on the AVN listserv at the time we sent our invitation. This number includes our team members (n=10) and our advisory group (n=8). Some of our advisory group members are also service providers who may have taken the survey. Of the 54 we reached out to, five (9.2%) responded to our request. All service providers completed the survey electronically, we sent no paper surveys via the mail. We gathered information relevant to our outreach efforts in addition to the data described below. These questions and prompts pertained to where service providers access information relevant to their work and what types of information they would be interested in receiving. Responses to these prompts are not included in this evaluation brief as they do not represent program evaluation, but rather will inform our outreach efforts moving forward.

Demographics

Participating service providers described their organizations as either ‘agricultural services’ (n=4) or non-profit (n=1). They work in rural (n=4) and urban (n=1) areas. Service providers who filled out this survey serve a single county (n=2), others serve a small number of neighboring counties (n=2), and one works statewide (n=1). Service providers who responded to our survey support clients with back injury (n=3), joint injury (n=2), cardiovascular disease (n=1), vision impairment (n=1), and mental health conditions/mental illness (n=1).

Service Providers’ Understanding and Perception of AgrAbility Virginia Programming

All but one respondent listed that they agree with the prompt “I am aware of what AgrAbility Virginia services are available to my clients” with one responding neither agree nor disagree. All respondents agreed that information provided by AgrAbility Virginia has been helpful to their clients, with three listing ‘agree’ and two listing ‘somewhat agree.’ All but one responded that they would recommend the AgrAbility Virginia program to other farmers or farm families for assistance, with one listing ‘neither agree nor disagree.’ When service providers were asked to provide their ‘biggest takeaway’ from what they have learned from participating in the AgrAbility Virginia Program and any additional comments, they provided these answers:

• “A very caring and professional staff.”

• “That there is help for the famers with disabilities."

• “Overall good program.”

• One requested “information about resources,” while another requested a “list of possible services.”

Conclusion

AgrAbility Virginia is designed to support farmers with disability, illness, and injury and promote safety, wellness, and accessibility on the farm through education, rehabilitative services, and assistive technology. We conducted a convergent parallel mixed methods design to explore our farmer clients’ experience within our program and evaluate whether and how we are meeting our service and education goals. The evaluation consisted of farmer surveys (n=9), service provider surveys (n=5), and farmer interviews (n=3). Farmers who participated in the survey and interview portions of our evaluation were overwhelmingly satisfied with their participation in the program, noting positive changes in their quality of life, as well as the accessibility and productivity of their farms. Though we had fewer responses from service providers (9.2% of those invited responded), service providers provided insight into the ways AgrAbility Virginia meets the needs of their farmer clients and the ways AgrAbility Virginia may expand engagement statewide.

References