Bot Tracking

Overview

CitedPro tracks visits from AI bots so you can see which AI systems are crawling your content. This helps you understand your visibility in the AI search ecosystem.

Enabling Bot Tracking

1. Go to CitedPro → Settings
2. Toggle Enable Bot Tracking to on
3. Save changes

What Gets Tracked

For each AI bot visit, CitedPro records:

• Bot Name: The identified bot (GPTBot, ClaudeBot, etc.)
• Category: AI Search Crawler, AI Agent, AI Data Scraper, etc.
• Page URL: Which page was visited
• Timestamp: When the visit occurred
• User Agent: The full HTTP user agent string
• IP Address: For identification and blocking purposes

Tracked AI Bots

CitedPro recognizes 40+ AI bots across these categories:

AI Search Crawlers

• GPTBot, ChatGPT-User (OpenAI)
• ClaudeBot, Claude-Web (Anthropic)
• PerplexityBot (Perplexity AI)
• Google-Extended (Google Gemini/Bard)

AI Agents

• Amazonbot (Amazon Alexa)
• Applebot-Extended (Apple Intelligence)
• Meta-ExternalAgent (Meta AI)
• Bytespider (ByteDance/TikTok)

AI Data Scrapers

• CCBot (Common Crawl)
• cohere-ai (Cohere)
• Diffbot (Diffbot)
• FacebookBot (Meta)

Viewing Analytics

Go to CitedPro → Analytics to see your AI bot traffic:

Summary View

• Total AI bot visits over selected period
• Breakdown by bot category
• Trend chart showing visits over time

Detailed View

• Individual visits with timestamps
• Most visited pages
• Bot frequency comparison

Data Retention

Bot tracking data is automatically cleaned up based on your retention settings:

To change retention (Premium only):

1. Go to CitedPro → Settings
2. Adjust the Data Retention setting
3. Save changes

Understanding the Data

What High Bot Traffic Means

Frequent visits from AI crawlers indicate your content is being indexed for AI search. This is generally positive — it means AI assistants can potentially recommend your business.

What Low Bot Traffic Means

If you're seeing few AI bot visits:

• Your site may be new or have few backlinks
• Check that robots.txt isn't blocking AI bots
• Verify your llms.txt and site-data.json are accessible
• AI bots may prioritize high-authority sites

Performance Impact

Bot tracking adds minimal overhead:

• Detection happens early in the request
• Only bot visits are logged (not regular traffic)
• Database writes are optimized
• Automatic cleanup prevents table bloat

If you experience performance issues, you can disable tracking in Settings.