CHAPTER 16

Evaluating Impact of Volunteer Programs

R. Dale Safrit, EdD North Carolina State University

This chapter introduces and defines the closely related concepts of evaluation, im-pact and accountability, especially as applied to volunteer programs. The author dis- cusses four fundamental questions that guide the development and implementation of an impact evaluation and subsequent accountability of a volunteer program.

Evaluation in Volunteer Programs

The concept of evaluation as applied to volunteer programs is not new. As early as 1968, Creech suggested a set of criteria for evaluating a volunteer program and con- cluded, “Evaluation, then, includes listening to our critics, to the people around us, to experts, to scientists, to volunteers so that we may get the whole truth [about our pro- grams]” (p.2). This approach to evaluation was well ahead of its time since up until the past decade, when authors within our profession either only addressed the evaluation of holistic volunteer programs superficially (e.g., Brudney, 1999; Naylor, 1976; O’Con- nell, 1976; Stenzel & Feeney, 1968; Wilson, 1979) or not at all (e.g., Naylor, 1973; Wil- son, 1981). Even in the first edition of this text, fewer than four total pages of text were dedicated to the topic of evaluation within chapters dedicated to other traditional vol- unteer program management topics, including recruiting and retaining volunteers (Bradner, 1995), training volunteers (Lulewicz, 1995), supervising volunteers (Brud- ney, 1995; Stepputat, 1995), improving paid staff and volunteer relations (Macduff, 1995), monitoring the operations of employee volunteer programs (Seel, 1995), in- volving board members (Graff, 1995), and determining a volunteer program’s success (Stepputat, 1995).

However, for volunteer programs operating in contemporary society, evaluation is a critical, if not the most critical, component of managing an overall volunteer program and subsequently documenting the impacts and ultimate value of the program to the

389 Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:20:43.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

target clientele it is designed to serve as well as the larger society in which it operates. As early as 1982, Austin et al. concluded that “Only through evaluation can [nonprofit] agencies make their programs credible to funding agencies and government authori- ties” (p. 10). In 1994, Korngold and Voudouris suggested the evaluation of impact on the larger community as one phase of evaluating an employee volunteer program.

The critical role of volunteer program impact evaluation in holistic volunteer man- agement became very apparent during the final decade of the twentieth century, and continues today (Council for Certification in Volunteer Administration, 2008; Merrill & Safrit, 2000; Safrit & Schmiesing, 2005; Safrit, Schmiesing, Gliem, & Gliem, 2005). While most volunteer managers understand and believe in evaluation, they most often have focused their efforts on evaluating the performance of individual volunteers and their contributions to the total program and/or organization. In this sense, evaluation has served an important managerial function in human resource development, the results of which are usually known only to the volunteer and volunteer manager. As Morley, Vinson, and Hatry (2001) noted:

Nonprofit organizations are more often familiar with monitoring and reporting such information as: the number of clients served; the quantity of services, programs, or activities provided; the number of volunteers or volunteer hours contributed; and the amount of donations received. These are important data, but they do not help nonprofit managers or constituents understand how well they are helping their clients. (p. 5)

However, as nonprofit organizations began to face simultaneous situations of stagnant or decreasing public funding and increasing demand for stronger account- ability of how limited funds were being used, volunteer program impact evaluation moved from a human resource management context to an organizational develop- ment and survival context. The volunteer administration profession began to recog- nize the shifting attitudes toward evaluation, and in the early 1980’s the former Association for Volunteer Administration (AVA) defined a new competency funda- mental to the profession as “the ability to monitor and evaluate total program results . . . [and] demonstrate the ability to document program results” (as cited in Fisher & Cole, 1993, pp. 187, 188). Administrators and managers of volunteer-based programs were increasingly called on to measure, document, and dollarize the impact of their programs on clientele served and not just the performance of individual volunteers and the activities they contribute (Safrit & Schmiesing, 2002; Safrit, Schmiesing, King, Villard, & Wells, 2003; Schmiesing & Safrit, 2007). This intensive demand for greater accountability initially arose from program funders (public and private) but quickly escalated to include government, the taxpaying public, and even the volunteers them- selves. As early as 1993, Taylor and Sumariwalla noted:

Increasing competition for tax as well as contributed dollars and scarce resources prompt donors and funders to ask once again: What good did the do- nation produce? What difference did the foundation grant or United Way alloca- tion make in the lives of those affected by the service funded? (p. 95)

According to Safrit (2010, p. 316), “The pressure on nonprofit organizations to evaluate the impact of volunteer-based programs has not abated during the first

390 Evaluating Impact of Volunteer Programs

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:20:43.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

decade of the new [21st] century, and if anything has grown stronger.” With regards to overall volunteer management, evaluation continues to play an important role in the human resource management of individual volunteers; most volunteer managers are very familiar and comfortable with this aspect of evaluation in volunteer programs. However, today’s volunteer managers are less knowledgeable, skilled, and comfort- able with the concept of impact evaluation as only the first (if important) step in meas- uring, documenting, and communicating the effects of a volunteer program immediately on the target clientele served by the organization’s volunteers, and ulti- mately on the surrounding community.

A Symbiotic Relationship: Evaluation, Impact, and Accountability

In the overwhelming majority of both nonformal workshops and formal courses I have taught, participants will inevitably use three terms almost interchangeably in our discussions of evaluating volunteer programs. The three concepts are symbi- otically linked and synergistically critical to contemporary volunteer programs, yet they are not synonymous. The three terms are evaluation, impact, and accountability.

Evaluation

Very simply stated, evaluation means measurement. We “evaluate” in all aspects of our daily lives, whether it involves measuring (evaluating) the outside temperature to determine if we need to wear a coat to work, measuring (evaluating) the current bal- ance in our checking account to see if we can afford to buy a new piece of technol- ogy, or measuring (evaluating) the fiscal climate in our workplace to decide if it is a good time to ask our supervisor for a salary increase. However, for volunteer pro- grams, “evaluation involves measuring a targeted program’s inputs, processes, and outcomes so as to assess the program’s efficiency of operations and/or effectiveness in impacting the program’s targeted clientele group” (Safrit, 2010, p. 318).

The duel focus of this definition on a volunteer program’s efficiency and effective- ness is supported by contemporary evaluation literature. Daponte (2008) defined evaluation as being “done to examine whether a program or policy causes a change; assists with continuous programmatic improvement and introspection” (p. 157). Royse, Thyer, and Padgett (2010) focused on evaluation as “a form of appraisal…that examines the processes or outcomes of an organization that exists to fulfill some so- cial need” (p. 12). These definitions each recognize the important role of evaluation in monitoring the operational aspects of a volunteer program (i.e., inputs and processes) yet ultimately emphasize the program’s ultimate purpose of engaging volunteers to help bring about positive changes in the lives of the program’s targeted audience (i.e., outcomes). These positive changes are called impacts.

Impact

Contrary to popular belief, volunteer programs do not exist for the primary purpose of engaging volunteers merely to give the volunteers something to do or for supplying an organization with unpaid staff to help expand its mission and purpose. Rather,

A Symbiotic Relationship: Evaluation, Impact, and Accountability 391

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:20:43.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

volunteer programs ultimately seek to bring about positive impacts in the lives of the targeted clientele the volunteers are trained to support either directly (through direct service to individual clients) or indirectly (through direct service to the service- providing organization). The latter statement does nothing to discount or demean the critical involvement of volunteers, but instead challenges a volunteer manager to con- tinually focus and refocus the engagement of volunteers on the ultimate mission of the sponsoring organization and the outcomes it seeks to bring about. In other words, it forces volunteer managers to identify and focus on the volunteer program’s desired impacts.

According to Safrit (2010):

Impact may be considered the ultimate effects and changes that a volunteer- based program has brought about upon those involved with the program (i.e., its stakeholders), including the program’s targeted clientele and their surrounding neighborhoods and communities, as well as the volunteer organization itself and its paid and volunteer staff. (p. 319)

This inclusionary definition of impact focuses primarily on the organization’s rai- son-d’être, and secondarily on the organization itself and its volunteers. Thus, it paral- lels and compliments nicely the earlier definition of evaluation as being targeted first toward the volunteer program’s targeted clientele, and secondly on internal processes and operations. Subsequently, volunteer managers must constantly measure the ulti- mate outcomes of volunteer programs, or stated more formally, evaluate the volunteer program’s impacts. However, merely evaluating a volunteer program’s impacts is not in itself a guarantee for the program’s continued success and/or survival; however positive, the knowledge gained by evaluating a volunteer program’s impacts are prac- tically meaningless unless they are strategically communicated to key leaders and de- cision makers connected to the sponsoring organization.

Accountability

Accountability within volunteer programs involves the strategic communication of the most important impacts of a volunteer program, identified through an evaluation pro- cess, to targeted program stakeholders, both internal and external to the organization. Internal stakeholders would include paid staff, organizational administrators, board members, volunteers, and the clientele served; external stakeholders include funders and donors, professional peers, government agencies and other legitimizers, and the larger community in which the organization operates.

Boone (1985) was the first author to describe the critical role of accountability in educational programs and organizations, and the previous definition is based largely on that of Boone, Safrit, and Jones (2002). Unfortunately, volunteer managers are sometimes hesitant to share program impacts even when they have identified them through an effective evaluation; they often consider such strong accountability as be- ing boastful or too aggressive. However, accountability is the third and final concept critically linking the previous concepts of evaluation and impact to a volunteer program’s or organization’s continued survival. Volunteer managers must accept the professional responsibility in our contemporary impact-focused society to

392 Evaluating Impact of Volunteer Programs

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:20:43.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

proactively plan for targeted accountability, identifying specific key stakeholders and deciding what specific program impacts each stakeholder type wants to know. This targeted approach to volunteer program accountability will be discussed in more detail later in this chapter.

Four Fundamental Questions in Any Volunteer Program Impact Evaluation

Evaluation is a relatively young concept within the educational world; Ralph Tyler (1949) is often credited with coining the actual term itself, evaluation, to refer to the alignment between measurement and testing with educational objectives. And there is no dearth in the literature of various approaches and models for program evaluation. Some models are more conceptual and focus on the various processes involved in evaluation (e.g., Fetterman, 1996; Kirkpatrick, 1959; Rossi & Freeman, 1993; Stuffle- beam, 1987) while others are more pragmatic in their focus (e.g., Combs & Faletta, 2000; Holden & Zimmerman, 2009; Patton, 2008). However, for volunteer managers with myriad professional responsibilities in addition to but including volunteer pro- gram evaluation, I suggest the following four fundamental questions that should guide any planned evaluation of a volunteer-based program.

Question 1: Why Do I Need to Evaluate the Volunteer Program?

Not every volunteer program needs to be evaluated. This may at first appear to be a heretical statement coming from the author of a chapter about volunteer program eval- uation, and theoretically it is. Pragmatically, however, it is not. Many volunteer pro- grams are short term by design, or are planned to be implemented one-time only. In contrast, some volunteer programs are inherent with the daily operations of a volun- teer organization, or are so embedded within the organization’s mission that they are invisible to all but organizational staff and administrators. Within these contexts, a vol- unteer manager must decide whether the evaluation of such a program warrants the required expenditure of time and human and materials resources. Furthermore, one cannot (notice that I did not say, may not) evaluate any volunteer program for which there are no measurable program objectives. This aspect of Question 1 brings us again to the previous discussion of volunteer program impacts: What is it that the volunteer program seeks to accomplish within its targeted clientele? What ultimate impact is the volunteers’ engagement designed to facilitate or accomplish?

Any and all volunteer program impact evaluations must be based on the measur- able program objectives targeted to the program’s clientele (Safrit, 2010). Such mea- surable program objectives are much more detailed than the program’s mere goal, and define key aspects of the program’s design, operations, and ultimate outcomes. A measurable program objective must include each of the following five critical elements:

1. What is the specific group or who are the specific individuals that the volunteer program is targeted to serve (i.e., the program’s clientele)?

2. What specific program activities will be used to interact with the targeted clientele group (i.e., the intervention that involves volunteers)?

Four Fundamental Questions in Any Volunteer Program Impact Evaluation 393

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:20:43.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

3. What specific change is the intervention designed to bring about within the tar- geted clientele group (i.e., program outcome or impact)?

4. What level of change or success does the program seek to achieve? 5. How wil

Our website has a team of professional writers who can help you write any of your homework. They will write your papers from scratch. We also have a team of editors just to make sure all papers are of HIGH QUALITY & PLAGIARISM FREE. To make an Order you only need to click Ask A Question and we will direct you to our Order Page at WriteDemy. Then fill Our Order Form with all your assignment instructions. Select your deadline and pay for your paper. You will get it few hours before your set deadline.

Fill in all the assignment paper details that are required in the order form with the standard information being the page count, deadline, academic level and type of paper. It is advisable to have this information at hand so that you can quickly fill in the necessary information needed in the form for the essay writer to be immediately assigned to your writing project. Make payment for the custom essay order to enable us to assign a suitable writer to your order. Payments are made through Paypal on a secured billing page. Finally, sit back and relax.

Do you need an answer to this or any other questions?

Order Plagiarism Free Answer Here

About Wridemy

We are a professional paper writing website. If you have searched a question and bumped into our website just know you are in the right place to get help in your coursework. We offer HIGH QUALITY & PLAGIARISM FREE Papers.

How It Works

To make an Order you only need to click on “Order Now” and we will direct you to our Order Page. Fill Our Order Form with all your assignment instructions. Select your deadline and pay for your paper. You will get it few hours before your set deadline.

Are there Discounts?

All new clients are eligible for 20% off in their first Order. Our payment method is safe and secure.

Hire a tutor today CLICK HERE to make your first order

Related Tags

Academic APA Writing College Course Discussion Management English Finance General Graduate History Information Justify Literature MLA