How accurate is the detector?
We publish how we measured the detector and where it fails. Read this before relying on a result.
Model
We run the open-source desklib AI-text detector (DeBERTa-v3-large, MIT licence) on our own servers. Text is split into windows; the document score is the token-weighted average and sentence scores come from overlapping windows.
Calibration on real academic writing
Thresholds were set on 583 human-written academic texts (arXiv abstracts, PubMed Central papers by non-native authors and student essays). "Likely AI" is set so that about 1% of those human texts cross it; "Uncertain" covers the band between 1% and 5%.
False alarms on held-out human texts
In a two-fold split, the share of held-out human academic texts labelled "Likely AI" was 1% and 0.7%.
Results by text group
| Group | Texts | Labelled "Likely AI" | Labelled "Likely AI" or "Uncertain" |
|---|---|---|---|
| Human: arXiv abstracts | 200 | 0.5% | 3.5% |
| Human: PubMed Central, non-native authors | 183 | 1.1% | 6% |
| Human: student essays | 200 | 1% | 5.5% |
| Human: Wikipedia (before 2022) | 155 | 2.6% | 8.4% |
| Human: general web text | 150 | 32.7% | 36.7% |
| AI: 2026 models, unedited | 195 | 96.9% | 99% |
| AI: RAID benchmark, unedited | 200 | 96.5% | 99% |
| AI: RAID benchmark, paraphrased | 200 | 84% | 92% |
| AI: rewritten by humanizer tools | 200 | 59.5% | 74.5% |
acad583-2026-10-22
Known limitations
- Text from the newest models, especially when prompted to write like a human, is often missed.
- Paraphrasing and "humanizer" tools lower the score substantially.
- Short texts are unreliable; under 50 words we do not give a result.
- Heavily edited, formulaic or translated human writing can be flagged.
- Thresholds are calibrated on academic writing. On general web text, 32.7% of human-written samples were labelled "Likely AI".
A result is an estimate, not evidence of misconduct. Never use it as the only basis for an accusation.