ARTICLES

3-16-2006

OUTCOMES: TO MEASURE OR NOT TO MEASURE?

If you work at a nonprofit organization with funding from state or foundation grants then you are familiar with following and delivering the objectives that have been outlined by the funders. Has your organization been proactive in asking the necessary, but difficult questions:

Did we accomplish what we set out to do?
Was there more we could have done?
What other needs can we help meet?
How can we improve our services to make a difference in more people’s lives?

Most groups find themselves asking these questions at some point. As times change, the needs of the community change and it makes sense to assess those changes. The only way to ensure that we are meeting the specific needs of those we serve is to establish a means of evaluating the effectiveness the organization and measuring the outcome of services provided.

One of the workshops at the NYC Grassroots Media Conference last month set out to ask media-makers these same questions. The workshop, Community Video: Personal or Political? From Limited Access to Transformative Communication seemed to strike a chord in its standing-room only audience. One of the panelists opened up the room to discussion by simply asking should grassroots media measure the outcome of their work? What followed was a heated debate between those that thought a strategic plan should be developed and followed with outcomes measured, and those that felt that measuring outcomes was a tactic that only corporations employed.

When it comes to communications, a company's bottom line is relatively easy to measure and usually comes in the form of dollars and cents—return on investment (ROI) However, measuring the effectiveness of nonprofits’ work depends upon many factors. How many people were educated? Informed? Served? Engaged? Activated? How much money was raised? Did legislative policy change? Corporate policy? Public opinion?

A new study examining the increasing importance of the Internet to nonprofits revealed a surge of online donations along with marketing and advocacy benefits from building an Internet constituency. The ENonprofit Benchmarks Study is the first of its kind to look at the overall effectiveness of nonprofits using the Internet to raise money, build e-mail lists, and influence political causes.

"Nonprofits have only recently begun using the Internet for marketing and fundraising purposes, so they have little history from which to benchmark success," says the Advocacy Institute's Jennifer Milewski, also a study co-author. "The ENonprofit Benchmarks Study finally gives nonprofits a tool they can use to measure their own online success and compare their success with that of other nonprofits active online."

"Industry benchmarks for online outreach are critical so nonprofits can measure their success objectively," says Kira Marchenese, Director of Internet Strategy for Environmental Defense. "Without anything to compare our results to, it's hard to tell whether we could do a better job."

Some key findings:

• More Donations Online: Nonprofits raised 40% more money online in 2024 than the year before, likely driven in part by the surge in online giving after the cataclysmic Asian tsunami.

• E-Mail Overload: The rates at which online constituents open their e-mails declined from 30% in 2024-2004 to 26% in 2024-2005 as supporters became overwhelmed with e-mail.

• Budgets Matter: Nonprofits with larger online budgets had better online programs, building larger e-mail lists, generating more online activism, and raising more money online.

• E-Mail Lists Growing: On average, the nonprofits studied more than doubled their existing e-mail lists over a 12-month period.

• Bad E-Mail Addresses: Yet, for most nonprofits, over a quarter of e-mail addresses on their list go bad each year, posing a challenge for organizations trying to grow e-mail lists quickly.

• Activism More Popular Than Donations: Not surprisingly, more e-mail subscribers took online political action than made an online donation — 47% vs. 6%.

The size of an organization is not necessarily the prime measure for success of communication program. Instead, the strategic use of funds and other resources to sell a nonprofit's message to legislators, business leaders, potential donors and the general public, using all the tools at one's disposal in conjunction with other communications media, is critical.

Measuring the success (and failure) of communications initiatives through proper tracking; number of print pieces distributed, number of people watching/screening video presentations, number of times PSAs are seen or heard on TV and the radio, e-mail message open and response rates, is required to maximize the benefits these tools have on a nonprofit's communications success. Measuring outcomes doesn't just start and stop at the communications level. According to a United Way study, Outcome Measurement: What and Why, demonstrating that a nonprofit makes a difference for people is just as important:

* Recruit and retain talented staff.
* Enlist and motivate able volunteers.
* Attract new participants.
* Engage collaborators.
* Retain or increase funding.
* Gain favorable public recognition.

Results of outcome measurement show not only where services are being effective for participants, but also where outcomes are not as expected. Program managers can use outcome data to:

* Strengthen existing services.
* Target effective services for expansion.
* Identify staff and volunteer training needs.
* Develop and justify budgets.
* Prepare long-range plans.
* Focus board members' attention on programmatic issues.

The NYC Grassroots Media Conference workshop ended on an interesting note. In a discussion of measuring the outcome of grassroots media, one participant who said she was from 'corporate media' and was just interested in all the 'buzz' about grassroots media, said "she didn't get it"—she didn't understand the message grassroots media was telling. That's really a shame, because grassroots media has tremendous power when used correctly. Perhaps it's a good idea to think about 5 simple public relations principles: who, what, where when, how. If we can apply this simple criteria with a corresponding metrics component to track ROI, a strategic plan, program, service or media initiative, then we are likely to be successful in developing an effective message to help achieve our objective— educating, engaging, empowering and getting people to act, with a donation or their voice.

"Outcome measurement has been a tremendous benefit to our agency by clarifying the services we provide. It has helped the staff, clients, board, constituents, and funders. We have a whole list of topics to talk about and we are all on the same page. This is concrete data. — Nina Waters, Executive Director, Practical and Cultural Education
(PACE Center for Girls)

—VOH