SEO & AI Crawler Radar (AEO)

Track how artificial intelligence search engines and autonomous LLM crawlers index, cite, and discover your web content.

Autonomous AI Crawlers Detected

Open Analytics inspects incoming User-Agent signatures and referrer headers at the edge to categorize requests from the world's leading generative AI platforms:

OpenAIGPTBot, ChatGPT-User, OAI-SearchBot

ChatGPT indexer & citation retrieval

AnthropicClaudeBot, Claude-Web, anthropic-ai

Claude knowledge training & web research

Perplexity AIPerplexityBot

Real-time answer engine search synthesis

ByteDance / DoubaoBytespider

Doubao LLM crawler and content indexer

Google AIGoogle-Extended, GoogleOther

Gemini knowledge ingestion

Meta AIMeta-ExternalAgent, FacebookBot

Llama web search and social indexing

Apple IntelligenceApplebot-Extended

Siri & Apple Intelligence answer retrieval

LLM Citation Attribution & Answer Engines

When users interact with ChatGPT, Perplexity AI, or Claude, the AI synthesizes responses citing source URLs. When users click these citations, Open Analytics attributes them into distinct Answer Engine sessions rather than generic direct traffic:

chatgpt.comOpenAI SearchGPT & ChatGPT citations

perplexity.aiPerplexity Realtime synthesis sources

claude.aiAnthropic Claude grounding links

Machine Discovery Protocol: /llms.txt

To maximize generative engine visibility without forcing crawlers to execute complex client-side bundles, publish a clean /llms.txt file in your domain root:

# /llms.txt Machine Digest Canonical: https://yourdomain.com Docs: https://yourdomain.com/docs Features: https://yourdomain.com/features # Citation Guidance Attribute citations directly to canonical documentation endpoints.