Empowering Participatory Evaluation - Building an Evaluative Culture at the Media School, University of the Arts London

This presentation is in Pecha Kucha format. The conference covers evaluation from all sectors (not just HE) and is the main membership body for evaluation professionals. The presentation outlines a three-year project on developing an evaluative culture in the teaching and learning community in the Media School at the University of the Arts London. Evaluation is central to understanding the effectiveness of teaching (Thomas et al., 2017; Thomas, 2020, Austen et al 2021). Office for Students and QAA provide extensive guidance on self-evaluation measures for institutions to assess for teaching quality; robust quantitative and qualitative evaluation was significant with TEF gold awardees (Moore et al 2023).

Yet evaluation processes are not always accessible for lecturers at the local level. A research project was set up to explore how to establish an evaluative culture within the Media School to embed educational enhancement within day to day teaching. The research involved institution case studies and research with staff and students.

This led to a set of criteria for effective evaluation and the design of an innovative, flexible evaluation toolkit for data collection that is easy for staff to implement in an on-going way. I focused on designing qualitative, inclusive, participatory tools to capture a wide range of student voices. A blog was created to present the tools and disseminate findings.

Three years on this framework is embedded into the school to assess anything from one-off interventions to course reapprovals; the 18 projects so far have covered around 700 students and 28 staff exploring themes like student confidence, experience, learning and community. Tools include things like collaborative journey mapping, advice cards and dashboards. Staff and students work in partnership through the process. For students it is reflective and meaningful part of their learning journey while for staff it is the opportunity to test innovation and pinpoint what works and build on that; we then share best practice.

The presentation will explore the key features that have helped build capability and create an evaluative culture:
• A consistent, robust approach for evaluation projects
• Funded supportive cross-school culture
• An embedded, participatory approach to capturing student voices
• Shared responsibility among staff
• The development of a thematic database of evidence
• Regular best-practice sharing events

Evaluation has led to an action-oriented approach to education enhancement. Building on theories of change and an iterative cycle (Thomas et al. 2017) each evaluation involves a statement of next steps and proposed actions, which in turn are evaluated. In developing evaluative mindsets, staff are using new tools for assessing their work, ask evaluative questions as they research and design the process into their teaching.

Takeaways from the session include:
• framework which empowers staff to implement their own evaluations
• practical, creative, varied data capture tools
• ways of sharing best practice and blog of resources
• characteristics of the evaluative culture

Evaluation can be a flexible and positive way to capture evidence of impact and effectiveness in HE. The innovative approach we have established has led to staff engagement, educational enhancements and improved conversations with students; through it we have been able to capture and celebrate staff achievements in their T&L innovations.