Sign up for the Today newsletter
Get everything you need to know to start your day, delivered right to your inbox every morning.
More than 1,000 MCAS essays scored incorrectly due to AI mishap
The temporary technical issue impacted an average of one or two students' scores in each of the 145 affected districts.
Approximately 1,400 MCAS essays were scored incorrectly due to an AI-grading mishap, according to the state Department of Elementary and Secondary Education (DESE).
A temporary technical issue caused the scoring issue, which impacted roughly 1,400 of 750,000 essays, or an average of one or two students’ scores in each of the 145 affected districts, according to DESE following reporting by NBC10.
“As one way of checking that MCAS scores are accurate, the Department of Elementary and Secondary Education releases preliminary MCAS results to districts and gives them time to report any issues during a discrepancy period each year,” a DESE spokesperson told Boston.com. “During this year’s discrepancy period, DESE learned that a technical issue led to some student essays not being scored accurately, and the issue was resolved in early August.”
Advertisement:
A teacher at the Reilly Elementary School in Lowell noticed a discrepancy between their student’s essays and their scores and notified a principal and district leaders in mid-July, which is within the discrepancy period, Assistant Superintendent Wendy Crocker-Roberge of Lowell Public Schools told Boston.com.
“Two essays appeared to be off by one and two points each, but the third essay, which was originally scored a zero out of seven, I re-rated as a six out of seven.” Crocker-Roberge said.
Once DESE was notified, the testing contractor, Cognia, investigated the issues and ensured rescoring of the affected essays. Additionally, score distributions of randomly selected batches of essays were given another look, according to DESE.
Advertisement:
“AI grading certainly has the potential to turn important student performance data around for schools expeditiously, which helps schools to plan for improvement,” Crocker-Roberge said.
The usage of AI to score MCAS essays is relatively new and scores the tests based on human-scored exemplar essays. After being scored by AI, 10% of MCAS essays are also scored by a human to compare scores and check for discrepancies, a DESE spokesperson said.
“With time, AI will become more accurate at scoring, but it is possible that the 90-10 ratio of AI to human scoring is not yet sufficient to achieve the accuracy rates desired for high-stakes reporting,” Crocker-Roberge said.
Full MCAS score reports typically become available for students and families in the fall, according to previous score release dates.
To comment, please create a screen name in your profile
To comment, please verify your email address
Conversation
This discussion has ended. Please join elsewhere on Boston.com