Underminer

Underminer aims to discourage AI model crawler bots and other unwanted bots, while allowing search engine crawlers and RSS aggregators to work normally.

It works in two ways:

• Using robots.txt rules to prohibit well-behaved AI training bots from crawling your site.
• By selectively corrupting the meaning of text content, to sabotage badly-behaved bots that ignore those rules.

Well-behaved search engine bots are permitted to crawl the site as normal, without being restricted by robots.txt or being subjected to sabotage.

Well-behaved LLM training bots that obey robots.txt will not receive any corrupted content.

Features

• 100% free and open source. No upsells or nags or subscriptions or promotions or freemium versions or donations. Ever.
• Auto-detects crawler bots.
• Completely invisible to real users and well-behaved bots.
• Verifies IP ranges of well-known bots to detect fake Googlebot crawlers etc.
• (Almost) zero-configuration.
• Language-neutral; works with page and post content in most languages.
• Preview mode: you can see what a bad bot will see.

Instead of blocking bad bots, they will simply be served corrupt and useless content:

• Words and sentences are randomly rearranged.
• Lists and paragraphs are re-ordered.
• Numbers are randomized.
• Currency symbols and measurement units are randomly changed.
• Alt text and descriptions are randomly switched around.
• Links are switched around.
• Image URLs are intentionally broken.

Well-behaved bots

For the purposes of this plugin, a well-behaved bot is one that:

• Obeys robots.txt.
• Publishes an up-to-date list of IP ranges.
• Identifies search crawling separately from AI model training crawling.

Requirements

• PHP 8.0 or higher
• WordPress 6.4 or higher