Scholarly insight Autumn 2017: a Data wrangler perspective

As the OU is going through several fundamental changes, it is important that strategic decisions made by Faculties and senior management are informed by evidence-based research and insights. One way how Data Wranglers provide insights of longitudinal development and performance of OU modules is the Key Metric Report 2017. A particular new element is that data can now also be unpacked and visualised on a Nation-level. As evidenced by the Nation-level reporting, there are substantial variations of success across the four Nations, and we hope that our interactive dashboards allow OU staff to unpack the underlying data. The second way Data Wranglers provide insight to Faculties and Units is through the Scholarly insight report series. Building on the previous two reports whereby we reported on substantial variation and inconsistencies in learning designs and assessment practices within qualifications across the OU, in this Scholarly insight Autumn 2017 report we address four big pedagogical questions that were framed and co-constructed together with the Faculties and LTI units. Many Faculties and colleagues have reacted positively on our Scholarly insight Spring 2017 report, whereby for the first time we were able to show empirically that students experienced substantial variations in success within 12 large OU qualifications. As evidenced in our previous report, 55% of variation in students' success over time was explained by OU institutional factors (i.e., how students were assessed within their respective module; how students were able to effectively transition from one learning design of one module to the next one), rather than students' characteristics, engagement and behaviour.

We have received several queries and questions from Faculties and Units about how to better understand these students' journeys, and how qualifications and module designs could be better aligned within their respective qualification(s). As these are complex conceptual and Big Pedagogy questions, in Chapter 1 we continued these complex analyses by looking at the transitional processes of the first two modules that OU students take, and how well aligned these modules and qualification paths are. In Chapter 2, we explored the more fine-grained, qualitative, and lived experiences of 19 students across a range of qualifications to understand how OU grading practices and (in)consistencies of assessment and feedback influenced their affect, behaviour, and cognition. In addition to building on previous topics, we introduced two new Scholarly insights in Chapter 3 and Chapter 4. As the OU is increasingly using learning analytics to support our staff and students, in Chapter 3 we analysed the impact of giving Predictive Learning Analytics to over 500 Associate Lecturers across 31 modules on student retention. Finally, in Chapter 4 we explored the impact of first presentations of new modules on pass rates and satisfaction, whereby we were able to bust another myth that may have profound implications for Student First Transformation.

Working organically in various Faculty sub-group meetings and LTI Units and in a google doc with various key stakeholders in the Faculties, we hope that our Scholarly insights can help to inform our staff, but also spark some ideas how to further improve our module designs and qualification pathways. Of course we are keen to hear what other topics require Scholarly insight.