ASSESSMENT METHODS: A STATISTICAL ANALYSIS OF OUTCOME EFFECTS AND IMPACT ON EQUALITY, DIVERSITY & INCLUSION

Abstract

The lifting of COVID-19 restrictions and concerns about use of artificial intelligence have led to various methods being adopted for university assessment. This paper conducts a statistical analysis of all final year module results at the Faculty of Laws of University College London in the main 2025 assessment period. The principal assessment methods used (with various percentage weightings for different modules) were: (i) in person, ‘closed-book’, timed written exams; (ii) in person oral exams; and (iii) ‘open-book’ essays. The research objective was to determine whether choice of exam format had any effect on outcomes and/or impact on equality, diversity and inclusion (as to which, possible concerns had been raised by staff and students). The available equalities markers in the Faculty of Laws’ data were gender and ‘Access UCL’ status, namely students admitted under a contextual offer scheme for UK-domiciled applicants from underrepresented groups at UCL. The methodology used was to run regressions on the data using Excel, controlling for various factors and interactions. The main findings and research outcomes were that, controlling for demographics, a higher percentage of grade assessed by written exams was associated with a lower grade outcome (significant at the 0.1 level). A 10 percentage point increase in written exam weighting was associated with a 0.4 point decline in overall grade. Controlling for assessment method, female candidates obtained a higher grade outcome by 1.1 points (significant at the 5% level) and ‘Access UCL’ students obtained a lower grade outcome by 1.5 points (significant at the 0.1% level). No significant interactions between demographics and assessment methods were observed. In the future data from other assessment periods could be added, in particular to see whether there are interactions between demographics and assessment methods with a larger sample size; and/or a negative coefficient found for oral examinations becomes statistically significant.