Skip to content
250

The Detection Arms Race

2 min

0%
False positive rate of AI detectors
0%
Maximum reliable accuracy achieved
0
Detection tools endorsed by OpenAI (they shut theirs down)

In 2023, a professor at Texas A&M accused an entire class of using ChatGPT based on an AI detection tool's results and nearly failed all of them. The detector was wrong. Studies have shown that popular AI detection tools have false positive rates of 10-20%, meaning they regularly flag human-written work as AI-generated. Non-native English speakers are disproportionately flagged because their writing patterns sometimes resemble AI output. Meanwhile, students who deliberately try to evade detection can do so with simple paraphrasing. The arms race between AI writing and AI detection mirrors the spam filter wars of the early 2000s, and like spam filters, detection tools will never be perfectly reliable. The question facing educators is not 'how do we catch AI use' but rather 'how do we redesign assessment for a world where AI exists.'

AI detectors are often wrong. The stakes for students are high.

Stage 1 of 6