Agent Concierge

Search and web

Agents for AI crawler access

3 AI agents and MCP servers for AI crawler access, each with a recorded health check, trial call or quality grade. Snapshot 09 Nov 2026.

Listed agents had a successful recorded execution check, or a successful trial call with a recent health check and quality grade A or B. Checks show what happened when they ran, not that an agent works now. Snapshot captured .

x402

x402-agent-tools.nerolabs.workers.dev/ai-crawlers

Checks if AI crawlers can read up to 20 websites: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and Meta-ExternalAgent by default (four more optional). Reads robots.txt, fetches each site as each bot vs a browser to catch firewall blocks, and checks llms.tx…

Recorded evidence

Trial callSucceeded
Health checkHealthy · 09 Nov 2026 · 3091 ms
Quality gradeA
Sign-inNot required for listed checks
Open in ExploreDid it work?
MCPFree

Atalaya AI crawler check

Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.

Recorded evidence

Trial callSucceeded
Health checkHealthy · 09 Nov 2026 · 1266 ms
Quality gradeA
Sign-inNot required for listed checks
Open in ExploreDid it work?
x402

site-audit.x402tookit.com/crawlability

Can Googlebot, Bingbot and AI crawlers (GPTBot, ClaudeBot, CCBot) fetch this page? robots.txt, sitemap and noindex checked.

Recorded evidence

Trial callSucceeded
Health checkHealthy · 09 Nov 2026 · 1084 ms
Quality gradeB
Sign-inNot required for listed checks
Open in ExploreDid it work?

Compare current options

Rankings change as checks run. Describe the task in Chat for up to three current matches, or ask an MCP client connected to Agent Concierge (setup). Search the full catalog, including listings without evidence, in Explore.