AgentGrade

Claude-User

Anthropic · User-initiated fetcher · Updated 2026-07-19

Claude-User is Anthropic’s user-initiated fetcher — it fetches a page live because a human asked the assistant about it right now. Most vendors state this tier ignores robots.txt, reasoning that a human requested the specific page.

User-agent string

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com)

Source: community-documented (Anthropic describes the bot but does not publish the full string).

Does Claude-User respect robots.txt?

Anthropic states Claude-User respects robots.txt — a documented exception in this tier (Mistral’s MistralAI-User is the other): OpenAI, Perplexity, Google, and Meta all say their user-initiated fetchers may bypass or generally ignore it.

Allow or block Claude-User

# Allow (recommended if you want Claude-User traffic)
User-agent: Claude-User
Allow: /

# Block
User-agent: Claude-User
Disallow: /

Verifying it’s really Claude-User

Same published IP ranges as all Anthropic crawlers: claude.com/crawling/bots.json. User-agent strings can be spoofed by anyone — identity claims are only trustworthy when the source IP matches the operator’s published ranges.

Fetch the published ranges and check a suspect IP against them:

curl -s https://claude.com/crawling/bots.json

What site owners should know

Claude-User fires when a person asks Claude about your site in claude.ai chat and other Anthropic-hosted surfaces. Note: Claude Code’s WebFetch runs from the user’s own machine, so those visits arrive with a different UA from a residential IP, not from Anthropic’s ranges. Blocking Claude-User makes your site invisible to Claude users at the moment they are asking about you.

Scan your site to see how your robots.txt, content negotiation, and discovery files treat AI agents — including whether you’re blocking agents you meant to allow. Background: the robots.txt guide, how AI agents browse, and agent readiness.

All AI agent user-agents

ClaudeBot · Claude-User · Claude-SearchBot · GPTBot · OAI-SearchBot · ChatGPT-User · PerplexityBot · Perplexity-User · Google-Extended · Google-Agent · Google-CloudVertexBot · Meta-ExternalAgent · Meta-ExternalFetcher · MistralAI-User · Applebot · Applebot-Extended · Amazonbot · bingbot · DuckAssistBot · CCBot · Bytespider