Are we making our community better?
For years RISE has measured progress by numbers.
How many people completed a programme.
How many new clients came to RISE in a year.
How many referrals came from the courts.
How many new staff have we hired to work with the increased number of clients.
Then there are the exit surveys of clients, where we measure progress in tick boxes.
Did our programme help you change your behaviour? Fully disagree, somewhat disagree, neutral, somewhat agree, fully agree.
Do you have new skills to help you with respectful relationships? Fully, somewhat, etc.
Did our programme help you understand how violence affects children?
Was the environment safe and supportive?
Have the changes I made helped my whānau?
Sometimes we ask open-ended questions and these give us really useful information. But do answers to these questions tell the whole story? They don’t. And we have come to realise this over the last few years.
We collect these numbers – the yeses and nos, the fully agrees or somewhat disagrees – to report to our funders. The Ministry of Justice, Ministry for Social Development, Oranga Tamariki, and the Department of Corrections are numbers-based people. We are reimbursed by some funders for hitting numbers targets and others give generic contracts but none ask us to report on community impact.
The numbers are important, but do they answer this key question: are we making a difference? Are we making our community better? Are we making the lives of the people who live in our community better?
Changing strategies to answer the right questions
A few years ago, our Board started to wrestle with these questions as part of a wider strategic planning process. It was clear that RISE had become more dependent financially on Government contracts over the past couple of decades. Government funding waxes and wanes depending on who is in charge. Creating stability and clarity for the only specialist family violence organisation in the Top of the South meant finding a new financial model.
The Board started to look at more diverse funding. With key local business leadership around the table, it was clear that RISE was doing good work, but could it prove that to potential funding partners? What is the best way to convince a funding partner we are a worthy investment? And for the long-term. How do we prove we are making a difference and how can we demonstrate that. The numbers and the tick boxes didn’t tell the whole story.
It was clear to all the Board members that we had to find another way to talk about our impact. And, it had to lead to a more progressive and sustainable funding model. That discussion led us to start work with The Ākina Foundation. They help other organisations define their positive social and environmental outcomes. This work began in 2022 and will continue this year and next year. Right now we’re halfway through and it’s a good time to pause and share our progress.
What do we know about people after they leave RISE?
When people complete a RISE programme, most of the time we don’t see them again. Do we know if the lessons and the tools are helping them? Do we know if they are better off one, three, or five years later? And what about their families – if a couple has stayed together or if they have separated, how is everyone in the extended family faring?
For the most part, we don’t know because we don’t stay involved with people as they navigate life after RISE. Yes, Nelson and Richmond and Tasman are small places so clinicians bump into former clients here and there. Anecdotal information is often heart-warming to share at a team meeting, but it isn’t going to help us prove the case that we are making a difference in the wider community.
What’s our impact?
With Ākina’s help, we have created an impact model and framework that demonstrates the impact we are having. What’s an impact model? It sets out the activities we undertake, the outcomes they lead to, and the ultimate impact that then happens after we have done our part. Graphic 1 illustrates a summary:
Graphic 1
RISE responds to family violence that exists in New Zealand. Family violence affects a range of stakeholders, from victims to whānau to community to perpetrators to workplaces to social organisations to government and right back to families. Our specialist non-violence and safety services are delivered with empathy and based in evidence. They are whānau-centred and aligned to best worldwide practice.
The work we do results in four key successes, detailed in summary in Graphic 1 and in detail in Graphic 2.
In order to show we are having this impact, we have identified measures to illustrate our effectiveness. The work we are doing now includes collecting the data to measure the above impacts. In addition, we are tracking third-party research that will assist us in making our case by documenting societal changes.
How we are changing our measurements
There are several points where we can begin to collect new, useful data to measure our sustained positive societal impact. At a new client’s initial assessment meeting, we are going to ask a few different questions of each person. Then we are going to ask that same question again three months after a person has started a programme and again six months after the person has started the programme. Finally, we’ll ask that same question in a follow up survey two years after the client has completed their programme.
For example, to measure if there has been a reduction in violence, what is the percentage of clients who report a reduction in the frequency of using violence? We will ask a new client at the initial assessment meeting how many times they have used violence in a given timeframe, ask again at three and six month intervals, and then ask again two years post programme. Collecting this data for each client over a period of time will tell a useful story that helps us understand the short- and medium-term impact.
Another way for us to measure our impact is for clinicians to make notes during programmes. For example, did a client share that they have used a RISE tool or new skill during one of their sessions? If a facilitator hears this, it will be noted and tallied. Does a client report during a session having a good understanding of their personal triggers? If so, the facilitator will note it and tally it.
We are also going to start measure the number of clients who self-refer to further RISE programmes and the number of past clients who make contact with facilitators/counsellors after completing a programme.
Our own data will be married with various Government, university, and NGO research studies as well as ACC and BERL research on economics to widen the lens on effectiveness.
Next steps
We have chosen a list of priority outcomes to get started. We’re modifying our administrative procedures and training our staff.
Keeping you updated is a priority too. If we are telling our story to you, we hope you will tell our story to others as well. The more we can share our successes about being effective in ending family violence in our community, the greater our collective impact will be.
We also aim to prove our worth to new funding partners in time. As we all know, Government funding is often fleeting and never long-term. While we know that we are doing the crucial work every day, it isn’t reflected in numeric governmental reports. Telling our story has been important, but individual stories don’t provide statistically significant evidence of success either.
And yet we know the work we are doing matters. We see families are healthier, that adults are mending and maintaining positive levels of wellbeing. And we know our communities are better for our part in this work. It will be so very rewarding to prove it!