NW 11 Educational Improvement and Quality Assurance Archives

Peer review in the era of generative AI models: An ethical call

by Dr Ayman Hefnawi | Emerging Researchers' Group, NW 11 Educational Improvement and Quality Assurance, NW 9 Assessment, Evaluation, Testing and Measurement

The emergence of generative AI models, such as ChatGPT, is significantly impacting various facets of society, including research and academia. Given their ability to generate human-like text based on input data or prompts, generative AI models have profound implications for the academic community. These implications entail ethical and societal challenges within the peer review process, raising questions about the potential role of such models (Schintler, McNeely and Witte, 2023).

As a reviewer, I believe that it is my responsibility to highlight and discuss these critical and timely ethical concerns. Thus, while acknowledging the potential benefits of using such models for academic purposes, this blog post aims to highlight the caveats of using such models in peer-reviewing, and some of their potential pitfalls. This, in turn, emphasises the fundamental need for clearly shared and regularly updated ethical guidelines that ensure the healthy use of such models in academia.

Can generative AI assist or replace expert reviewers?

How much of AI is too much? This raises the ethical dilemma: should generative AI assist or replace expert reviewers? To address such fundamental questions, we should first acknowledge that AI, like any technology, is designed to increase the productivity of professionals, not necessarily replace them.

To begin with, generative AI (GAI) can assist reviewers, particularly those for whom English is not their first language, in producing clear and concise reports in less time. For instance, generative AI tools, such as editGPT, have the potential to save time in assessing text readability; tasks that are less intellectually demanding (Checco et al., 2021). I therefore believe it is acceptable for expert reviewers to use generative AI to streamline the review process, while emphasising the necessity of checking the report accuracy.

The pitfalls of AI in the peer review process

Generative AI models are typically unable to offer recommendations based on the latest research findings in the dynamic and complex field of education. The expertise of human peer reviewers is generally beyond the capabilities of generative AI models, which lack the required domain knowledge and intellectual capacity, at least in the foreseeable future. These limitations may have serious implications where GAI models typically provide general comments that lack critical content about the concerned manuscript (Donker, 2023); this means the lack of proper improvement recommendations with the possibility of the manuscript being unjustly dismissed.

Generative AI models may produce reviews that contain ethical concerns and biases. AI algorithms risk copying and/or possibly expanding human biases (Schintler, McNeely and Witte, 2023). An example of one ethical consideration, which even human reviewers may not fully adhere to, is the importance of respecting the authors’ perspective and not converting their manuscript into that of the reviewers. AI might make recommendations that do not really respect or consider the authors’ perspective enough. This could potentially result in humans being responsible not just to fellow humans but also to machines (Schintler, McNeely and Witte, 2023). Reviewers equally need to be aware of caveats such as breaching the confidentiality of the manuscript under review, as generative AI models may use or share the original ideas as part of their machine-learning processes (Mollaki, 2024). These ethical issues clearly highlight a call for the wise use of unfolding AI technology.

Effective GAI Implementation

We acknowledge that GAI technology is developing at a fast pace (Checco et al., 2021), so it is not easy to predict its exact capabilities. Accordingly, we might witness the emergence of generative AI models that address some of the above ethical concerns. Therefore, it is the collective responsibility of all involved in knowledge production (authors, reviewers, editors, academic supervisors) to continually review and update the scientific community, as well as society members in general, on best practices for using generative AI in academia.

Peer reviewers have an ethical duty to uphold their full responsibility and resist the temptation to simply delegate the job to generative AI models. While researchers importantly advocate for policies that govern the use of AI in peer review (Mollaki, 2024), I do believe that it is, first and foremost, an ethical responsibility that should be central to the review process, with clear repercussions for those disregarding these fundamental ethics.

Thus, this blog post, following Facer and Selwyn (2021), advocates for a ‘non-stupid’ optimism that acknowledges the limitations of using digital technologies in academia. This necessitates that the dialogue be positioned within continuous academic discussions and research on reliable and ethical AI-powered peer review. All those involved in advancing educational research need to ensure that ethics is at the heart of the knowledge production process; otherwise, the integrity of the entire process would be compromised.

Key Messages

Generative AI models like ChatGPT have critical implications for academic peer review.
Expert reviewers play a crucial role in maintaining the quality and integrity of the peer review process.
Generative AI should complement, not replace, human judgment and expertise in academia.
Continuous review and dialogue are necessary to ensure ethical and effective use of AI in peer review.

Dr Ayman Hefnawi

Mathematics Instructor, ADVETI, UAE

Ayman Hefnawi holds a Doctor of Education from the University of Bath, United Kingdom, and a master’s degree in educational leadership and management from the University of Warwick, United Kingdom. Additionally, Ayman serves as a reviewer for educational journals and maintains memberships in various academic associations

https://www.researchgate.net/profile/Ayman-Hefnawi
https://www.researchgate.net/profile/Ayman-Hefnaw
https://twitter.com/aymanhefnawi
https://orcid.org/my-orcid?orcid=0000-0002-7744-6997
https://www.scopus.com/authid/detail.uri?authorId=57315619100

References and Further Reading

Checco, A., Bracciale, L., Loreti, P., Pinfield, S. and Bianchi, G., 2021. AI-assisted peer review. Humanities and Social Sciences Communications, 8(1), pp.1-11. https://www.nature.com/articles/s41599-020-00703-8

Donker, T., 2023. The dangers of using large language models for peer review. The Lancet Infectious Diseases, 23(7), p.781. https://www.thelancet.com/journals/laninf/article/PIIS1473-3099(23)00290-6/fulltext?rss=yes

Facer, K. and Selwyn, N., 2021. Digital technology and the futures of education: Towards ‘Non-Stupid’optimism. Futures of Education initiative, UNESCO.

Mollaki, V., 2024. Death of a reviewer or death of peer review integrity? The challenges of using AI tools in peer reviewing and the need to go beyond publishing policies. Research Ethics, p.17470161231224552. https://journals.sagepub.com/doi/10.1177/17470161231224552

Schintler, L.A., McNeely, C.L. and Witte, J., 2023. A Critical Examination of the Ethics of AI-Mediated Peer Review. arXiv preprint arXiv:2309.12356. https://arxiv.org/abs/2309.12356

Improving the quality of education – EERA Network 11 through the years

by Dr Samuel Gento | 30 Years of EERA, NW 11 Educational Improvement and Quality Assurance

EERA is celebrating 30 years in 2024, and as part of our anniversary celebrations, we have invited people who have been at the heart of the association to share their memories and reflections. In a series of blog posts, which will run throughout 2024, we will share those precious memories, from the people who helped foster the global EERA community.

In this blog post, Dr Samuel Gento, Honorary Link Convenor of Network 11, reflects on the activities of the network to improve the quality of education in Europe and beyond.

The celebration of the 30th Anniversary of the European Educational Research Association (EERA) is an important event to be enjoyed by all of us who are concerned with the study and improvement of education in Europe and in the rest of the world. Education is a fundamental factor in humanity’s progress and development of all human beings, wherever they are. In general, the contributions of EERA extend to different areas and responsible people, such as the following:

Quality of education in itself;
Professors from universities and other educational institutions;
Educators, in general (in their various positions and intervention modalities);
Researchers, in general, and, especially, those dedicated to the improvement of education;
Universities and other educational institutions;
Educational systems in countries around the world;
Progress of all countries, consolidated by a good education;
Interpersonal human relationships, optimized with a good educational level;
Democratic organizations, strengthened by their members’ good education.

Although the European Educational Research Association (EERA) has its headquarters and its priority field of activities in Europe, the impact of its research extends to all areas and countries of our planet, as has been demonstrated by participants from other continents, such as Asia, Africa, America, Australia and Oceania.

EERA Network 11 – part of the EERA family

In my role as convenor and link-convenor of Network 11, Educational Improvement and Quality Assurance, I have received the invaluable help provided by the Association, in general, and, in particular, by its components such as:

• The EERA Office in Berlin
• The Executive Board, with its President, General Secretary, Treasurer and their staff members
• The EERA Council
• The co-convenors, especially those from Network 11
• The reviewers of proposals, proceeding from all continents
• The presenters of papers, symposia, posters, round tables, etc.
• The participants in the different sessions, especially those of Network 11

My first presentation at the Association Conferences (ECER) took place in 1999 at the Conference held in Lahti (Finland).

After successive presentations, I was designated as ‘link convenor’ of Network 11 in 2007, at the ECER held in Ghent (Netherlands), a position previously implemented by Dr. Jan van Damme, from Holland).

With the support of Network co-convenors, I acted as Link Convenor’ until 2014, when I presented my resignation at the ECER held in Porto (Portugal).
In 2018, I was designated ‘Honorary Link Convenor, a position I continue to hold today.

After my resignation as Link Convenor, other co-convenors were appointed to this position in Network 11:

Linn Grant McMahon, from Scotland (from 2015 to 2016)
Heidi Flavian, from Israel (from 2016 to 2021)
Ineta Luka, from Latvia (from 2021 to 2024)

The purpose and activities of Network 11 over the years

The fundamental purpose of Network 11 has been to improve quality of education at its different levels and modalities. To this aim, we propose to investigate the procedures and strategies that contribute to a constant improvement of education. This will have an impact on an improvement in situations and conditions of every person, both in their personal sphere, as well as in the social, economic and stability ones in their particular context.

In the activities carried out by Network 11, we have received the aforementioned aid of EERA. In implementing interventions in conferences (ECER) we have had, in particular, the collaboration of Network 7, Social Justice and Intercultural Education, and Network 26, Educational Leadership.

We have had to overcome some challenges in operating Network 11, such as the designation of successive link-convenors. This difficulty results from the increased responsibility and dedication that the coordination of the network requires. We also changed the name of the network itself. The initial name of ‘educational effectiveness’ was changed to ‘educational improvement’, by understanding that the quality of education requires a continuous and endless process of continuous improvement.

Beyond EERA – Network 11 in the educational research community

In addition to the activities of network 11 within EERA, we have carried out some initiatives aimed at research and improvement of education in various contexts. Among them, we may mention the following:

The creation of the ‘Leadership and Quality in Education’ network www.leadquaed.com
The constitution of the ‘European Association of Leadership and Quality in Education’, registered in the Spanish Ministry of Internal Affairs
The collaboration with several European universities for the delivery of a Master’s Degree (Universities of Spain -UNED-, Latvia, Germany, and the Czech Republic)
The promotion of the Journal of Quality in Education
The publication of the book From Pedagogy to Quality Assurance in Education: An International Perspective, 2020, Heidi Flavian (ed.), Emerald Publishing Limited
The publication of various articles in education journals
Participation in conferences held in different countries of Europe and America.

Dr Samuel Gento

Emeritus Professor at the UCJC (University of Camilo José Cela), in Madrid

Brief professional and academic history

Primary School teacher in rural and urban schools.
Secondary Education teacher in different provinces in Spain.
Education Inspector of the Spanish Ministry of Education.
Secretary of Inspectorate Counsel in Burgos and Madrid.
Associate Professor at UNED (National University of Distance
Education), in Madrid.
Lecturer at the UNED.
Full Professor at the UNED.
Emeritus Professor at the UCJC (University of Camilo José Cela), in Madrid.
Doctor Honoris Causa from the University of Latvia (in Riga).
Link Convenor from Network 11 of the EERA.
Honorary Link Convenor of Network 11 of EERA.
Founder and President of the ‘European Association of
Leadership and Quality of Education’ (registered at the Spanish Interior Ministry).
Honorary President of the ‘European Association Leadership and Quality of Education’.

University academic positions

Curriculum Coordinator of the UNED.
First Vice Dean at the Faculty of Education of UNED.

Lines of research and professional development

Member of Spanish Delegation in international conferences and seminars, especially those organized by UNESCO.
Coordinator of research projects on: organization of educational institutions, teaching of foreign languages (especially English) and in-service teacher training (particularly in special education).
Professor of Didactics, School Organization, Educational School Supervision, and Special Education.
Promoter and director of the ‘European Joint Master on Educational Treatment of Diversity (120 ECTS)’, imparted by UNED (University of Distance Education) and Universities of Latvia (in Riga), Karlova (in the Czech Republic) and Reutlingen (in Germany).
Coordinator and publisher of 24 modules of Official Master’s Degree of Educational Treatment of Diversity.
Counsellor of the ‘State Education School Council’.
Presentations at conferences, seminars and courses in different Universities in Latvia, the United States and Latin America (especially in Argentina, Bolivia, Costa Rica, Mexico and Venezuela).
Presentations at international conferences, mainly those organized by the ‘AERA (American Association of Research in Education)’, ‘EERA (European Association of Research in Education)’, ‘World Curriculum Council’, ‘EADTU (European Association of Distance Teaching Universities)’ and ‘EDEN (European Distance Education Network)’.
Organization of 11 International face to face Conferences on ‘Educational Treatment of Diversity’ in Madrid and Palma de Mallorca (Spain), and 2 virtual ones.
Organization of the First International Conference on ‘Leadership and Quality of Education’.
Member of the scientific committee of education journals such as: Bordón (from the Spanish Society of Pedagogy), Participación Educativa (from the State Education School Council) and ‘Quality Assurance in Education’ (from the United Kingdom).
Founder and president of the ‘European Association of Leadership and Quality of Education’.
Participation in: English teacher training, evaluation of educational institutions, improvement of the quality of institutions, educational leadership and educational treatment of diversity.
Publications in books and scientific magazines on topics such as: organization of educational centres, quality of educational institutions, indicators and predictors of quality of educational institutions, pedagogical or educational leadership, special education, educational treatment of diversity, therapeutic pedagogy, adult training, educational inspection, school supervision, management and supervision of educational centres.