A new study finds teachers tend to provide assistance to similar subsets of students when using AI-powered educational tools, rather than touching base regularly with everyone in their classes. The findings could be used to develop tools that help teachers track their classroom interactions to ensure they are giving each student the attention they need.

“AI-powered tools are increasingly common in K-12 classrooms, but teachers still play a critical role,” says Qiao Jin, first author of the study and an assistant professor of computer science at North Carolina State University. “For this study, we wanted to examine how teachers who use AI-powered tools determine which students need help – and how those teachers actually distribute their time among their students.”

For this study, the researchers looked specifically at teachers using intelligent tutoring systems (ITS) to teach middle-school math. ITS are AI-powered software that responds to student activity to provide customized assistance through hints and feedback, as well as tracking student performance.

For the first part of the study, researchers interviewed nine middle school math teachers who used ITS in their classrooms. The interviews helped researchers understand how the teachers determine which students require an intervention (a teacher visit) and what kind of help the teachers provide.

“While teachers said it would be ideal to spend one-on-one time with every student, they noted that this is not possible,” Jin says.

Instead, the teachers made decisions about who to help based on many factors. Two of the most significant factors were whether a student had required assistance in the past, and a student’s “engagement state.”

“ITS can notify teachers when students have been consistently entering incorrect answers or have not interacted with the system for an extended time,” Jin says. “Those are engagement states called ‘struggle’ and ‘idle,’ respectively. And either of those engagement states might lead a teacher to touch base with the relevant students.”

To see how these teacher behaviors are reflected in practice, the researchers drew on data covering 1,437,055 interactions between students and an ITS. The data covers 339 students enrolled in 14 middle and high school math classes across 10 U.S. schools during the 2022-23 school year. All of the data the researchers looked at is data that the relevant teachers had access to via their ITS dashboards.

“We found that teachers are more likely to interact with students that they have interacted with before, even after considering who is engaged and disengaged in the classroom,” says Jin. “Basically, if a teacher has intervened to help a student in the past, they are more likely to intervene to help that student in the future.

“Teachers have their own definitions of fairness and their own understanding of student needs, based on their training and experiences,” says Jin. “We believe our findings can be used to develop software tools, such as dashboard features, that support teachers by giving them information they can use to make decisions about how they allocate their time in a way that is consistent with their definitions of fairness and student need.

“Teachers have a difficult job and developing better tools to help them do that job effectively is worthwhile.”

The paper, “Sticky Help, Bounded Effects: Session-by-Session Analytics of Teacher Interventions in K-12 Classrooms,” will be presented at the 16th Annual Learning Analytics & Knowledge Conference (LAK26) being held April 27-May 1 in Bergen, Norway. The paper was co-authored by YiChen Yu of NC State; and by Conrad Borchers, Ashish Gurung, Sean Jackson, Sameeksha Agarwal, Cancan Wang, Pragati Maheshwary and Vincent Aleven of Carnegie Mellon University.

The work was done with support from the Institute of Education Sciences of the U.S. Department of Education, under grant R305A240281.