Are Assessment Practices Well Aligned Over Time? A Big Data Exploration

In the last twenty years a range of approaches have been adopted to facilitate Assessment of Learning as well as Assessment for Learning. With the increased interest in measuring learning gains using assessment data, it is important to recognise the potential limitations of using grades as proxies for learning. If there is a lack of alignment in terms of grade descriptors between modules within a qualification, students might perform really well on one module, and may underperform in a module that has relatively "harsh" grading policies. Using principles of Big Data, we explored whether students' grade trajectories followed a consistent pattern over time based upon their abilities, efforts, and engagement in two distinct studies. In Study 1, we explored a relatively large dataset of 13,966 students using multi-level modelling, while in a more fine-grained Study 2 we focussed on the pathways of students choosing their first two modules in six large qualifications. The findings indicated substantial misalignments in how students progressed over time in 12 large qualifications in Study 1. In Study 2, our analyses provided further evidence that students' grades did not seem to be well aligned. In all qualifications we found a highly significant effect of change over time depending on the achievement group. Based upon these findings, we provide clear recommendations how institutions might use similar insights into big data, and how they may improve the longitudinal alignment of grading trajectories by using consistent grading policies.