Results from a Survey on Post-Primary Teachers’ Experiences with Calculated Grading during COVID-19

In May 2020, as a result of Covid-19, the high stakes assessment at the end of post-primary education in Ireland (the Leaving Certificate Examination – LCE) was cancelled replaced by a system of calculated grades. In documentation sent to schools, the Department of Education and Skills (DES) made it clear that a calculated grade would result from the combination of two data sets:

• an overall percentage mark and ranking in each subject awarded to each student by their teacher (the school-based estimation process)
• data on past performance of students in each school and nationally (the standardisation process)

Following the issuing of results to students and the completion of the appeals process, an online questionnaire survey was conducted in the final months of 2020 by researchers at the Institute of Education, Dublin City University, with the aim of investigating how teachers’ engaged with the calculated grades process in their schools.  Data from a total of 713 respondents were used in a report published by the Centre for Assessment Research, Policy and Practice in Education (CARPE) on April 15^th 2021. This report is now available to download from www.dcu.ie/carpe.  The following are some highlights from this report.

Assessment Evidence Used

Teachers considered many different types of formative and summative assessments when estimating mark and ranks for their students. Particularly important were final year exams prior to lockdown (98%) and final year continuous assessments (92%). Four out of every five teachers indicated that knowledge of how previous students had performed in the LC influenced their decision-making.  Significantly, 88% said that formative assessments were important also. One respondent noted:

Personally, I feel very competent in assigning the predicted grades to my LC students in 2020 since I had assessed their performance in detail over a 2-year period…. Each exam/ portfolio/homework was assigned a weighting and a record of their performance updated to our Schoology platform. Students could readily assess their own progress over this period and all this data enabled a solid predicted grade for each student.

Teachers’ Reflections on the School-Based Estimation Process

At least 90% of teachers indicated that they were able to apply the DES calculated grades guidelines strictly when estimating marks and ranks for the majority of their students. However, some reported experiencing difficulties in adjudicating marks at grade boundaries.  For example, 61% said that they gave 5% or more of their students the benefit of the doubt and gave them a mark that moved them above a grade boundary, with 21% saying that they should have awarded a failing mark but didn’t.  One-third of respondents said that they awarded a higher mark for 5% or more of their students because they thought the national standardisation process might bring the student’s grade down.  While 73% said that the moderation process to align grades within their schools worked well, 26% reported raising a mark and 17% lowering a mark following engagement in the process. Significantly, the vast majority of teachers (92%) felt that the marks they awarded were fair.

Other Reflections

One in three respondents added commentary at the end of the questionnaire, with many focusing on the stress brought about by the fact that they lived in the same small communities as the students they were grading. Many identified parents, school management, media and politicians as sources of the pressure they felt.  One teacher expressed it thus:

I believe that while it would be ok for more teacher involvement in urban centres, the nature of rural and small town Ireland made the entire process very uncomfortable and I am sure that teachers will feel the rippling exponential impact of this for some time.

A number of events that transpired following the submission of school data to the DES were also highlighted as problematic.  The fact that the DES provided students with their rank order data came as a surprise to teachers and caused great disquiet. The removal, in late August, of school historical data from the standardisation process, following controversy about its use for calculated grades in the UK, was a source of great annoyance, especially among those working in high achieving schools. That said, some teachers noted that calculated grades had been an acceptable option in the context of a pandemic and that many students benefited from the fact that the grades awarded in 2020 were the highest ever.

Conclusion

The implementation of calculated grades in Ireland was a historic event as, for the first time since the introduction of the LCE in 1924, post-primary teachers engaged in the assessment of their own students for certification purposes. While difficulties arose, all those involved worked diligently to ensure that the class of 2020 could progress in their education and/or careers.  In 2021, Irish teachers will be asked to engage in a similar process while at the same time they will be preparing their students to take the traditional LC examinations.  The plan is that the two assessment systems will run side-by-side, and students will be given the option of choosing their best result in each subject.  Our hope is that findings from this survey will be useful to all those responsible for overseeing and implementing this challenging task.

References and Further Reading

Doyle, A., Z. Lysaght and M. O’Leary. 2021. Preliminary Findings from a Survey of Post- Primary Teachers Involved in the Leaving Certificate 2020. Calculated Grades Process in Ireland. Dublin: Centre for Assessment, Research, Policy and Practice in Education (CARPE), Dublin City University. Accessed April 15, 2021. https://www.dcu.ie/sites/default/files/inline-files/calculated_grades_2020_preliminary_findings_v2_2.pdf

Doyle, A., Lysaght, Z., & O’Leary, M. 2021. High stakes assessment policy implementation in the time of COVID-19: The case of calculated grades in Ireland. Irish Educational Studies, 40. DOI: 10.1080/03323315.2021.1916565 https://www.tandfonline.com/doi/full/10.1080/03323315.2021.1916565