Teaching

Lack of reliability : AI grading can only be inconsistent, due to the generative nature of LLMs.Lack of accuracy: AI cannot assess abstract competencies. Even though LLMs were trained to follow instructions, assigning quantitative scores to student works based on qualitative rubrics is a highly-complex and specific task they were never prepared for.Lack of relationship: Even if it could be made reliable and accurate, AI grading would mean the loss of an important way teachers get to know their students and build connections with them.Skill loss: By delegating grading to AI, teachers would lose familiarity with the targeted demonstration of learning that should guide instruction.Lack of agency, (or transparency): Using AI to grade, teachers would model poor use of this technology to their students by entrusting it with final decision-making and relinquishing their own thinking and responsibility. Or, even worse, they would not be transparent about it

general purpose technology

ETS’s researchers had asked GPT-4o to score the essays cold, without showing the chatbot any graded examples to calibrate its scores. It’s possible that a few sample essays or small tweaks to the grading instructions, or prompts, given to ChatGPT could reduce or eliminate the bias against Asian Americans. Perhaps the robot would be fairer to Asian Americans if it were explicitly prompted to “give out more perfect 6s.”