Turnitin: AI writing detection tool

The Centre for Education and Innovation (CEI) has been testing the new AI detection tool from Turnitin which flags text that may have been written by AI. Here is a comprehensive guide to how to use it effectively.

What is it?

Turnitin has a new AI writing detection tool that helps flag text that may have been generated by AI. It appears as a percentage in an AI indication button for staff and is not visible to students. The AI writing detection tool was released on 5 April, and no universities were able to access the tool prior to this release date. The Centre for Education and Innovation (CEI) is investigating its strengths and weaknesses.

Guidance in interpreting and utilising the tool

The tool provides an overall percentage that represents the proportion of the work that the Turnitin tool is confident was written by generative AI (with 98 per cent confidence). Turnitin advise the percentage shown in the AI writing detection indicator and AI writing report should be used with caution. It does not indicate academic misconduct, and further review of the submission should be undertaken.

It also highlights the text that it believes was AI generated. This is separate from the text matching for the Similarity report. The highlighted text may not be a full representation of the actual AI generated text; it does not indicate purportedly AI generated text to the same degree of accuracy as it does with text matching.

Testing suggests that there may be both false positives (relatively rare) and false negatives (more common). This tool may be useful as an initial flag to indicate that generative AI may have been used. However, due to the risks of false positives, it should not be used as the sole evidence to forward an allegation of potential academic misconduct. Also note that there are false negatives, so a low percentage does not necessarily mean that there is no AI generated text.

Currently, Turnitin has not provided the capacity for staff to download this report as a pdf. More information about using the tool is available in LEO guides under ‘Detection indicator’.

What action should I take?

Under ACU’s Student Academic Integrity and Misconduct Policy, unauthorised use of generative AI in work submitted for assessment is a form of academic misconduct. Due to the risk of false positives and negatives, this tool should be used in conjunction with other features of the submitted work that may indicate unauthorised use of AI, including:

• fabricated references
• fabricated quotes
• incorrect information
• the material being slightly off topic, repetitive or having the wrong focus.

If you see a work that Turnitin flags as being AI generated, always look for other features that may indicate AI has been used. Visit Academic Integrity FAQs for a more comprehensive list.

What support is available?

• LEO guides have been updated to provide guidance.
• Turnitin have provided resources, including a FAQ section.
• A recent Staff Bulletin article provided suggestions around assessment design to help mitigate the impact of generative AI tools and also summarized assessment design and feedback resources available to staff at ACU.

Contact Academic.Integrity@acu.edu.au if you require additional support.