Page Modified v1.0.4
Display the results of your Page Modified crawls right within your WordPress Admin.
Tags / crawler
68 WordPress plugins tagged “crawler”. Every result links to a crawlable PlugIndex profile.
Display the results of your Page Modified crawls right within your WordPress Admin.
Generate and serve a dynamic, cached /llms.txt AI context index. An add-on that works in partnership with Rankal SEO Optimizer.
Dynamic robots.txt for Multisite! Change parameters, or don't, for each site of your network without losing the defaults from WordPress Includes.
With SEObot Monitor for Googlebot you will be able to dump the server logs to Google Analytics, easily and automatically. This will allow you to dete …
Official Trackbase Analytics plugin for WordPress. Add your tracking ID and enable server-side crawler analytics in minutes.
SEO & GEO bot analytics: see Googlebot, ChatGPT, Claude and every AI crawler at work on each post, page and term — full crawl history included.
Serve AI crawlers clean Markdown of your content on the same URL, and auto-generate llms.txt / llms-full.txt (with an on-disk fallback).
Tracks and logs AI, Search and Social bots including ChatGPT, Perplexity, Googlebot, Bing and Meta crawlers.
Detect and control AI crawler access with local logging and history.
Track AI crawler visits such as GPTBot, ClaudeBot, and PerplexityBot server-side and visualize them in WordPress.
Manage AI crawler access. Block GPTBot, Claude & AI bots in robots.txt, generate dynamic llms.txt files, and track referral visits locally.
See how AI reads your site. Detects, verifies, and displays AI crawler visits — all data stays on your own server.
See which AI assistants — ChatGPT, Claude, Perplexity, Google and more — crawl your site, broken down by assistant, crawler, purpose and page.
Choose which AI bots can read your content, and publish an llms.txt file for AI search tools.
Observe human and machine traffic, identify suspicious automation, and optionally quarantine aggressive scrapers.
Detect AI crawlers, apply access policies, and understand automated access to your WordPress content.
See AI crawler visits and AI assistant referrals on your site, with optional signed sync to Cleotic.
Block unauthorized AI crawlers from scraping your content and monetize verified bot access.
Block GPTBot, ClaudeBot, Bytespider and 170+ AI crawlers and scrapers, bot by bot. Blocks by user agent and robots.txt. Optional daily list updates.
All-in-one management toolkit for WordPress: site security, bot blocker, crawler, image optimizer, variation fixer, affiliate manager, live streaming …
Protect WooCommerce filtered URLs from crawl floods with noindex/canonical controls, signed cookies, rate limits, logs, and rollback.
Verify supported AI bot identities, allow legitimate agents, and block spoofed claims.
Monitor search engine bots visiting your WordPress website in real time.
Detect LLM user agents, enforce Google parity, and serve enhanced content through a configurable content service.