Introducing Project Evaluation: How can it help our voices to be heard? [Video + Transcript]

Alongside my academic research and other arts and activism, I work for an advocacy organisation in Leeds called Advonet. Part of this role has involved being a work-based evaluator for a very exciting endeavour – the Leeds LGBTQ+ Health Inclusion Project. This is a project that uses self-advocacy/peer support, and the training of healthcare professionals to address the exclusion and inequalities experienced by members of LGBTQ+ communities who have “additional needs” (i.e. who are autistic, have a learning disability, and/or who experience mental health difficulties).

My role as evaluator involves observing the project as it develops – sitting in on self-advocacy workshops and training sessions, and talking to project staff, and participants, to find out what they think about its achievements. In order to do this, it is essential that I am able to build a rapport with all those involved with the project, but I’ve found a problem with this, as it’s been difficult to communicate what my role involves. This has meant that project staff are unsure how to introduce me, and people are slightly concerned as they don’t know what to expect from me. This is especially challenging, given the personal, sensitive nature of much of the discussion that takes place in the workshops and training sessions. I needed to come up with a way to communicate what I’m trying to achieve.

Alongside this, I’m keen to start a conversation within the voluntary sector more widely – to discuss how we evaluate ourselves: how we celebrate our many achievements, and how we make our work even better.

So – I came up with the idea of making a video presentation (using PowerPoint and YouTube), in order to communicate my ideas around project evaluation – and, particularly, how we might use evaluation as a tool to ensure that all those involved in our projects and partnerships have the opportunity to have their “voice” heard in taking the work forward.

This style of presentation/communication itself is a work-in-progress. Things I’m aware of so far include the fact that the video is too long overall, perhaps because I’m communicating to multiple audiences. I think it may be difficult to hold people’s attention for so long. I’m also aware that in places, the text on the slides doesn’t sync with what I say in the audio. This is because as I was figuring out my working method, I edited things considerably. In future, I think I’ll be much clearer about what I’m doing, and about how much content I’ll be aiming to include altogether, meaning things will be much more fluently presented.

I’d really like to know how else I might develop these kinds of communication tools (both conceptually and technically).

Here is the video. I’ll copy the transcript below:

Transcript

Slide 1

Introducing Project Evaluation: How does it help our voices be heard.

Welcome to this short film, providing an introduction to project evaluation– and explaining the evaluation of the Leeds LGBTQ+ Health Inclusion Project.

This film is for anyone involved with the project – including project facilitators, peer supporters, self-advocates, and funders; along with anyone interested in carrying out project evaluations for other, similar, projects.

A transcript of what I am saying is available in the “notes” section of the PowerPoint file, or in the description box below the video, if you’re watching on YouTube. If you have any questions, contact details are provided at the end of the film – feel free to get in touch.

Slide 2

Hi,

I’m Gill Loomes, project evaluator for the Leeds LGBTQ+ Health Inclusion Project.

As a disabled person, I care about disabled voices being heard, and as a researcher, I want to use my skills to make that happen.

In this short film, I explain what project evaluation is, particularly how it’s being done at the Leeds LGBTQ+ Health Inclusion Project, – including how you can get involved, and how it can help us to be heard…

Slide 3

The word “evaluation” can have different meanings. But, in broad, simple terms, it means something like “checking the progress or merit of a thing, according to agreed standards”. When it comes to projects like the Leeds LGBTQ+ Health Inclusion Project, this is often thought to mean checking whether the project is delivering “value for money”.

You might think that the process of evaluating a project, such as the Leeds LGBTQ+ Health Inclusion Project, sounds like a rather boring job, that produces pages of words and numbers that no one will ever read. Or you might see it as obligatory – something we have to do, in order to keep the organisations that fund our projects happy. And it is often the case that funders require projects to evaluate their progress – for example, the Leeds LGBTQ+ Health Inclusion Project is funded by the Government Equality Office, which requires all projects that it funds to provide evaluations – in order to check how funding is being spent, and also to provide information that might be useful to other, similar projects.

But, depending on how it’s designed and carried out, a project evaluation can serve another important purpose – one that should matter a lot to advocacy, and self-advocacy organisations: a project evaluation can be an opportunity for all those involved in the project to have their say, and to have their voice heard – about how the project is run, about the difference it makes in society, and about what is working, and what is not working.

In this way, an evaluation can be an opportunity to promote equality within a project, and to ensure that everyone’s voice is heard. As it says here – [reads quote above from House, 1993]

So, let’s take a look at how a project evaluation can work – and how the evaluation of the Leeds LGBTQ+ Health Inclusion Project aims to ensure that the voices of everyone involved in the project are heard.

Slide 4

There are 2 more slides in this film. They explore the following questions [reads questions on slide]

Slide 5

So, the first question we are going to think about is “What is Project Evaluation – and how can it help our voices to be heard?”

We can think about what an evaluation is by answering 3 questions:

How (is the evaluation done) – an evaluation looks at something systematically – that is, it’s organised, and happens according to a plan.

What (does the evaluation do) – it produces information about a project, and the effects of the project.

Who (is the evaluation for) – anyone who is interested in the project; how it works, and how it can be improved. 

We can think about what an evaluation considers:

Project goals: How has the project met their goals? Have their goals changed in response to the priorities of project participants?

Decision-making: How does decision-making happen within the project? Is it efficient? Who gets a say in important decisions? Who is responsible for making sure things get done?

Different explanations: When evaluating a project, how do you ensure that different views about the project and how it works are represented?

Different users: When planning what questions to ask in an evaluation, think about who is going to use the evaluation. What do project participants need to know about the project? What do project funders need to know? How do you ensure you ask, and answer the right questions?

And we can think about what is important to make sure that an evaluation helps voices to be heard:

The evaluation must be responsive to the needs of project participants.

It can be goal-flexible, ensuring that different goals prioritized by participants are represented.

It ensures that alternative explanations and perspectives are represented and explored.

And it considers the needs of all users of the project, and the evaluation.

On the next slide, we’ll see how the evaluation of the Leeds LGBTQ+ Health Inclusion Project considers each of these things in its evaluation.

Slide 6

The LGBTQ+ Health Inclusion Project works to improve access to healthcare services for members of LGBTQ+ communities who are autistic, who have a learning disability, and/or who experience mental health difficulties. It does this by organising self-advocacy courses, and peer-support for members of these communities, and by offering training, designed and delivered by people with lived experience of these issues, to healthcare professionals.

The Government Equality Office, which provides the funding for the project , needs an evaluation of how the project works, and what it achieves. It needs to be able to show how public money has been spent.

The evaluation also provides an important opportunity for participants to have their say about what has worked, and what they might do differently.

And we also hope that the evaluation will be useful to other people and organisations looking to set up similar projects, so that they can learn from our experiences.

In order to provide information about the project that will be useful to all the people who are interested in the evaluation, and to ensure that we provide all the information that is needed, the evaluation is in two parts: An independent evaluator (called Tia) will look at the project from the “outside”, focusing mostly on how the project has achieved the goals it set out to achieve.

Alongside the independent evaluation, I am undertaking a work-based evaluation to give participants an opportunity to have their say about their experiences of the project.

If you are involved in the Leeds LGBTQ+ Health Inclusion Project, you might well come across Tia and me at some point. We may pop into a workshop you are involved in – this is to see how the workshops run, and to look at what works well, and what might be organised differently. And it is to give people an opportunity to describe their experiences with the project.

A word about what evaluation is NOT:

It is NOT to record details of discussions that are taking place, or to “check up” on anyone involved in the project – and anyone involved in the project with any questions or concerns should feel free to speak to Tia or me, at any time.

Contact details for me, and for Advonet (the organisation hosting the project) are on the next slide)

Slide 7

Thank you for listening to this short film. Please feel free to get in touch about the evaluation, or about the Leeds LGBTQ+ Health Inclusion Project itself.

Thanks so much for checking out this blog – I’d love to hear your thoughts about communicating ideas, research findings, evaluations, anything…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.