The validity of automatic AI detection is currently impossible to guarantee. No software will detect AI-generated content with 100% accuracy. AI detectors miss AI-generated content quite often. At the same time, it is not uncommon for them to flag human-written content as AI-generated. Both types of errors can be highly problematic and consequential.
Furthermore, AI-detecting software in its present state often raises equity and privacy concerns. Recent research suggests that AI detectors are more likely to flag content that was created by non-native English writers. Studies are underway to see if there is a similar bias against neurodivergent writers.
Turnitin. As of November 2023, in response to concerns about false positives and the inability to guarantee the validity of AI detection, the Advisory Board of the Office for Faculty Excellence recommended to discontinue the use of AI detection via Turnitin for the assessment of student submissions. The Academic Affairs Council supported that recommendation.
The AI detection functionality within Turnitin, originally available to Montclair Canvas users, was officially discontinued on November 20, 2023.
This decision was made following similar ones by many other universities, including Vanderbilt, Michigan State, Northwestern, and the University of Texas at Austin. Turnitin has acknowledged that its product may have a higher error rate than was initially suggested.
Other tools. At this point, we advise against uploading or pasting student writing into third-party tools and platforms not supported by the University, as it is likely to cause a major breach of student data safety. If you choose to use one of such third-party tools, it’d be safer to obtain students’ consent first. Also please bear in mind that, regardless of the tool you use, the results you get are not going to be fully reliable.
______________________________________________________________________________
Solutions
If you suspect that one of your students’ submissions is AI-generated, we recommend the following:
- check manually for AI writing red flags;
- consider the student’s broader context and writing history.
- talk to the student about their submission
- talking in-person or on zoom first is usually more productive than starting the conversation via email.
- enter the conversation from a non-assuming standpoint: most of the time, it is hard to know whether the student really used AI to create their submission — even if you are using an automatic AI detector. You can start the conversation by asking if there’s anything the student would like to share with you regarding that submission, and take it from there.
- discuss with the student the thought process that led to their submission.
- ask highly specific questions about the submission, targeting the parts of the submission that alarmed you.
Further guidance on responding to instances of suspected academic dishonesty is found on the OFE webpage Academic Dishonesty and Student Cheating under the “Strategies that Work” section.
Last Modified: Monday, April 8, 2024 7:07 pm
CC