Short answer
ChatGPT cites pages its search step retrieves; Gemini cites pages from Google Search grounding; Claude cites pages from its own web search tool. To get cited, allow OAI-SearchBot, Googlebot and Claude-SearchBot, make sure you're indexed, answer the question in plain text near the top of the page, add specifics like numbers, comparisons and sources, and get named on the third-party pages already cited for your category.
Citations vs mentions
A mention is your product’s name in the answer text. A citation is a URL the assistant links as a source. They’re related but not the same:
- Your own page can be cited without your product being recommended (say, a blog post cited for a definition).
- You can be recommended without your site being cited, because a cited listicle names you. For recommendation questions, this is often the more valuable case.
That’s why it helps to look at citations per page: of all the answers that cited a given page, how many named you? A page cited in 12 answers that named you in 0 of them is a clear outreach target.
How ChatGPT, Gemini and Claude choose sources
None of the three companies publishes a ranking formula. What is documented:
- ChatGPT search relies on OpenAI’s search crawler. OpenAI says OAI-SearchBot is used “to surface websites in search results in ChatGPT’s search features” and that sites opted out of it “will not be shown in ChatGPT search answers”.
- Gemini uses grounding with Google Search: the model writes search queries, reads the results and returns the sources it used.
- Claude uses Anthropic’s web search tool: Claude decides when to search, the API runs the searches, and the final answer cites its sources (Anthropic says citations are always on for web search). On the crawler side, Anthropic says Claude-SearchBot “navigates the web to improve search result quality” and that blocking it may reduce your visibility in search results.
- Google’s AI features (AI Overviews, AI Mode) need nothing special: a page must be indexed and eligible to show with a snippet, and Google says there are “no additional technical requirements”.
In practice: being findable by normal search is the entry ticket. Being the clearest, most specific answer to the question is what gets you picked from the candidates.
Crawler and indexing checklist
| User agent | Owner | Used for | If you block it |
|---|---|---|---|
OAI-SearchBot | OpenAI | ChatGPT search | Not shown in ChatGPT search answers |
GPTBot | OpenAI | Training foundation models | Content not used for training; search is governed separately |
ChatGPT-User | OpenAI | Visits a user asked for | OpenAI says it isn't used to decide search inclusion |
Googlebot | Google Search, incl. its AI features | Out of Google Search and AI Overviews / AI Mode | |
Google-Extended | Gemini training and grounding in Gemini Apps / Vertex AI | No effect on Google Search inclusion or ranking, per Google | |
Claude-SearchBot | Anthropic | Search result quality for Claude users | Not indexed for search; may reduce visibility |
Claude-User | Anthropic | Pages fetched when a person asks Claude | May reduce visibility in user-directed search |
ClaudeBot | Anthropic | Collecting content for model training | Future content excluded from training |
If you want to be cited, the minimum is:
User-agent: OAI-SearchBot
Allow: /
User-agent: Googlebot
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: Claude-User
Allow: /You can block GPTBot (training) and still appear in ChatGPT search. Blocking Google-Extended opts you out of Gemini grounding in the products Google lists, so leave it allowed if Gemini citations matter to you. Anthropic works the same way: blocking ClaudeBot opts out of training, while Claude-SearchBot and Claude-User are separate tokens with their own rules. Anthropic also says IP blocking isn’t a reliable opt-out, and each subdomain needs its own robots.txt. OpenAI notes robots.txt changes take about 24 hours to apply.
Write pages that are easy to cite
- 01
Answer first
Put a direct two-to-three sentence answer under the heading, using the words of the question. Details after.
- 02
Use question-shaped headings
“Is X free?”, “X vs Y”, “Does X work offline?” mirror how people ask assistants.
- 03
Add specifics
In the GEO research, adding quotations, statistics and cited sources gave the biggest gains, about 30–40% relative on the paper’s main visibility metric. Prices, limits, supported platforms and dates are the app equivalent.
- 04
Use real tables for comparisons
A comparison in an HTML table is easier to extract than one spread across paragraphs or baked into an image.
- 05
Keep it in text and keep it current
Google recommends keeping important content in text. Show an updated date and change it only when the content changes. Structured data is fine, but Google says no special schema is needed for its AI features and markup must match visible text.
Earn citations on pages you don’t own
For “best X” questions, the most cited pages are usually other people’s. Use your tracking data (or an MentionReach scan; see pricing) to list the domains cited most for your questions, then work through them by type:
- Lists and roundups: email the author with accurate, specific facts about your app and why it fits the list’s reader. Ask for consideration, not a link.
- Directories and launch sites: submit, and fill every field with the same positioning sentence you use on your site.
- Communities: answer questions where your app genuinely fits, say you built it, and follow each community’s rules.
- Review sites: for SaaS, ask real customers for reviews; for apps, prompt for store ratings at a natural moment.
Frequently asked questions
Does blocking GPTBot remove my site from ChatGPT?
No. OpenAI says GPTBot is for training foundation models. Whether you appear in ChatGPT search answers is controlled by OAI-SearchBot, which you can allow separately.
Does blocking ClaudeBot remove my site from Claude?
No. Anthropic says ClaudeBot collects content for training. Claude-SearchBot (search indexing) and Claude-User (pages fetched when someone asks Claude) are separate robots.txt tokens; blocking those is what may reduce your visibility in Claude's answers.
Do I need an llms.txt file to be cited by AI?
Google says no new machine-readable or AI text files are needed to appear in its AI features, and neither OpenAI's nor Anthropic's crawler documentation mentions llms.txt. It isn't a documented way to get cited.
Does schema markup help AI citations?
Google says there's no special schema.org markup needed for AI Overviews or AI Mode, and that structured data should match the visible text. Normal structured data still helps search engines understand the page.
Why do Gemini citations show Google redirect links?
Through the Gemini API, grounding sources often come back as Google redirect URLs with the page title. Follow the redirect, or use the title and domain, to see which page was actually cited.
Sources
- Overview of OpenAI crawlers · OpenAI
- Grounding with Google Search · Google AI for Developers
- Web search tool · Claude Docs (Anthropic)
- Does Anthropic crawl data from the web, and how can site owners block the crawler? · Anthropic Help Center
- AI features and your website · Google Search Central
- Google's common crawlers (incl. Google-Extended) · Google Search Central
- GEO: Generative Engine Optimization (Aggarwal et al., KDD 2024) · arXiv