Concerns grow in Boston University’s College of Communication as Turnitin, an AI and plagiarism detection tool, seem to produce a higher-than-reported false positive rate for student submissions.
COM students and staff recently took action, despite Annie Chechitelli, Turnitin’s chief product officer, writing that there was a false positive rate lower than 1% in April 2023.
Lisa Liberty Becker, a master lecturer for COM’s writing program, said these detectors are not always “super reliable” in the classes she teaches.
“As with all technology, it’s kind of a race to release things,” Becker said. “I don’t know whether Turnitin released something that was not quite ready.”
Becker said she assumed Turnitin’s AI detection tool was deleted because her department noticed it wasn’t always accurate.
“It was pretty clear immediately that it was not really a reliable tool, but I was just curious why it disappeared,” Becker said. “I assumed that it disappeared because everybody knew that it was unreliable, and it was probably causing more problems than it was worth.”
Becker said she was aware AI detection was not dependable after many students saw inaccurate percentages on their assignments that were quickly disproven.
“I would see percentages that looked high, but then when you go in and look at the writing and just talking to students in class, it’s clear that this was not something they used ChatGPT for,” Becker said.
Dr. Kenneth Lutchen, BU provost and chief academic officer ad interim, announced the creation of a University-wide task force focused on creating the “best practices and shared approaches around the use of generative AI at BU.”
Becker said professors across the University have different policies regarding the use of AI in the classroom and on assignments.
Sofia Harris, a sophomore in COM and the president of COM student government, said faculty might be concerned about students taking advantage of the lack of AI detection.
“I understand their hesitancy to inform us just because they don’t want students to abuse the current situation of not having AI detection,” said Harris. “But, I think, just for transparency, they should expose this information.”
Harris said plagiarism is a frequent problem on campus and professors should continue to use AI detection, but still understand that AI detection can be wrong sometimes.
“I feel like professors should be open to further discuss an unjustified situation with their students before taking any harsh reaction towards their grades,” Harris said. “I would advise COM to talk to other BU colleges to explore different AI detection programs or even investigate what other prestigious universities around the country use.”
Several universities around the country, like Vanderbilt and Northwestern, disabled Turnitin’s AI detection tool citing that although it’s important to mitigate unethical AI usage, it’s also important to be mindful of its benefits in the classroom environment, according to the Universities’ websites.
However, Becker said she found no success with AI detection tools after testing them out with her own writing and receiving incorrect outcomes.
“There hasn’t been an AI detection tool that’s been shown to be accurate,” said Becker. “I would put in a piece of my writing, the same piece, just to experiment, and every tool would give a different percentage.”
Danielle De La Espriella, a junior in COM and the College of Arts and Sciences, said that by giving more in-class assignments, professors can diminish the impact AI detection will have on their education.
“I believe COM should implement in-class assignments that allow for professors to familiarize themselves with their students’ writing,” De La Espriella said. “Knowing your students’ writing could be a way to decipher whether the flagging is credible or not.”
Similarly, Becker said she thinks it is “important for instructors to be familiar” with ChatGPT and other AI tools in order to understand what they are going against and “make their expectations clear.”
“I don’t think that the lack of a good detector is a concern just because the quality of the writing that we’re looking for in COM cannot be produced by a tool at this point in time,” Becker said.
