Article citationsMore >>

Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023). Unlocking the opportunities through ChatGPT tool towards ameliorating the education system. BenchCouncil Transactions on Benchmarks, Standards, and Evaluations, 3(2), 100115.

has been cited by the following article:

Article

The Theoretical and Practical Implications of OpenAI System Rubric Assessment and Feedback on Higher Education Written Assignments

1College of Business and Information Technology, South University, Savanah, USA

2Dissertation Department, Westcliff University, Irvine, USA


American Journal of Educational Research. 2024, Vol. 12 No. 4, 147-158
DOI: 10.12691/education-12-4-4
Copyright © 2024 Science and Education Publishing

Cite this paper:
LaJuan Perronoski Fuller, Christa Bixby. The Theoretical and Practical Implications of OpenAI System Rubric Assessment and Feedback on Higher Education Written Assignments. American Journal of Educational Research. 2024; 12(4):147-158. doi: 10.12691/education-12-4-4.

Correspondence to: LaJuan  Perronoski Fuller, College of Business and Information Technology, South University, Savanah, USA. Email: lfuller@southuniversity.edu

Abstract

Integrating artificial intelligence (AI) in teaching and assessment is becoming increasingly common. However, concern exists regarding the reliability and consistency of generative AI grading and feedback in higher education. This study aims to investigate AI chatbots, such as ChatGPT and Claude, and their ability to apply consistent grading and feedback. The research revealed implications of applying these OpenAI systems outside their intended purpose as language models for generating human-like text. The data collected reveal significant discrepancies in grading patterns, feedback rationale, and formatting of responses. These inconsistencies challenge some traditional theories of learning and assessment. For example, ChatGPT applied a 24-point difference between the lowest and highest scores (74%-98%) on the same assignment. Claude's lowest and highest scores revealed a 33-point difference. Each OpenAI system provided feedback that was less likely to promote learning due to inconsistent rationale per rubric item. Educators are encouraged to exercise caution when utilizing OpenAI as a grading and feedback system. Educators should rely on the expertise of a subject matter expert to ensure accuracy and fairness in assessment practices. By understanding the limitations of OpenAI grading and feedback, educators can mitigate potential unfair and inconsistent assessments to optimize student success and learning outcomes.

Keywords