Article

AI Detection Tools Are Astrology for Academics (And I Can Prove It)

Lorenzo Colombani

2026-01-16

4 min read

Infographic stating ‘AI Detectors: Unscientific & Unreliable,’ with three panels explaining that AI detectors fail scientific standards, give inconsistent results, and rely on the myth of ‘pure AI’ content — This is AI-generated. Or is it?

Picture this: You’re a student. You stayed up until 3 AM writing an essay about the French Revolution. You bled for this paper. You even read primary sources.

Your professor runs it through an AI detector.

“87% probability of AI-generated content.”

Welcome to the Twilight Zone.

Part 1: The Unfalsifiable Trap (Or: How AI Detectors Learned to Think Like Fortune Tellers)

Let me introduce you to Karl Popper. Brilliant philosopher. Magnificent mustache. No wait, that’s Nietszche. Popper was clean shaven. Anyway. In 1934, he asked a simple question: How do we know if something is actually science?

His answer was elegant: You have to be able to prove it wrong.

🧪 The Science Test

Table comparing statements by whether they can be proven wrong, showing a falsifiable scientific claim versus an unfalsifiable ‘fortune cookie’ statement.

Now let’s play a game I call “What Will the AI Detector Say?”

You write a clear, polished essay:

“Aha! Too polished. Classic AI. Look at those em dashes.”

You write a clear essay without em dashes:

“Suspiciously clean. They probably used a humanizer tool.”

You write something with typos and awkward sentences:

“They told the AI to write badly. Or they wrote it with AI and THEN downgraded it. Classic move.”

🎰 Congratulations! You’ve entered the AI Detection Casino, where the house always wins.

No matter what you write, the detector can claim AI involvement. You literally cannot prove it wrong.

You know what else works this way? Horoscopes. Conspiracy theories. That guy who insists he “totally knew” the plot twist in every movie.

Part 2: The Diagnosis From Hell (A Medical Horror Story)

Time for a field trip to 19th-century medicine! Don’t worry, we’ll keep our limbs.

Doctors back then diagnosed something called “General Paresis of the Insane” (GPI). It was a very serious, very official disease with:

✓ Distinct symptoms (grandiose delusions, tremors)

✓ Predictable progression (always fatal)

✓ High diagnostic agreement (doctors could spot it reliably)

Sounds legit, right?

Plot twist: It wasn’t a disease at all.

A French physician named Jean-Alfred Fournier did some detective work. He noticed that men collapsing with “GPI” in their 40s had syphilitic sores in their 20s.

GPI was just late-stage syphilis wearing a top hat and monocle.

Doctors had been reliably diagnosing something that didn’t exist.

The Two Tests Every Diagnostic Tool Must Pass

Table evaluating AI detectors, showing they fail reliability (tests disagree) and validity (the concept of ‘pure AI content’ doesn’t exist).

Part 3: The Sistine Chapel Problem (Or: Nothing Is 100% Anything)

Quick quiz: Who painted the Sistine Chapel ceiling?

If you said “Michelangelo,” you’re… partially correct!

Michelangelo didn’t:

Build the scaffolding
Chisel away the old decoration
Mix his own paints
Work alone

Workers, assistants, and centuries of accumulated artistic knowledge made that ceiling possible. Michelangelo was the creative director of a massive collaborative project.

Now here’s the question AI detectors can’t answer:

At what percentage of “assistance” does something stop being “your work”?

Used spell check? 🤔
Used a thesaurus? 🤔
Read other authors for inspiration? 🤔
Had a friend proofread? 🤔
Used AI to brainstorm? 🤔
Used AI to write everything? 🤔

The line these tools claim to detect is completely arbitrary. It’s like asking whether a soup is “homemade” if you used store-bought broth.

There is no bright line. There never was.

The Final Boss: A Summary

AI detection tools fail three fundamental tests of scientific legitimacy:

Table listing falsifiability, reliability, and validity requirements, showing that AI detectors fail all three by being unfalsifiable, inconsistent, and based on the nonexistent idea of ‘pure AI content.

AI detectors aren’t bad at their job.

They don’t have a job.

They’re elaborate random number generators with a veneer of scientific authority. Dowsing rods for the digital age. They’re the equivalent of determining guilt by whether someone floats in water.

The next time someone waves an AI detection score at you, remember:

A confident diagnosis of something that doesn’t exist is still nonsense.

🎤 Mic drop.

Now if you’ll excuse me, I need to go defend this article against accusations that an AI wrote it. Wish me luck.

Lorenzo Colombani translates between humans and machines — Certified Lawyer (France), court mediator, builder of AI instruments. Hannover, Germany.

Entity document: https://github.com/LorenzoColombani/lorenzo-colombani

Portfolio: https://lorenzocolombani.github.io · LinkedIn: https://www.linkedin.com/in/locolombani/

Want to know more about the author, Lorenzo Colombani, and his work in AI? Check out his website: https://www.lorenzocolombani.com