Answer engine optimisation (AEO) is the practice of making web content eligible, accessible and understandable to AI answer engines so that it can be selected and cited inside AI-generated answers. It is an industry term rather than a standard: no search provider or standards body publishes a formal definition, and the providers themselves describe what they require in their own documentation. This page explains what AEO covers, which signals Google, Microsoft, OpenAI and Perplexity actually document, how to measure appearances in AI answers with the providers' own tools, and which common claims have no official support. It is written for publishers, marketers, web teams and anyone asked to "do AEO" who wants to separate documented facts from sales talk. Providers are listed alphabetically where they appear together; order is never a ranking.
Last checked: 2026-11-15.
This is an independent reference page from NHI Mgmt Group based on public provider documentation; it is not sponsored or endorsed by any provider.
Key takeaways
- Eligibility comes before everything. Google says a page must be indexed and eligible to be shown with a snippet to appear as a supporting link in AI Overviews or AI Mode, and OpenAI says a site must not block OAI-SearchBot to appear in ChatGPT search summaries.
- Google documents no special AEO technique. Its Search Central page states there are no additional requirements or special optimisations for AI features, and no special schema.org structured data to add.
- Controls are now split by purpose. Search inclusion, training and grounding have separate switches: OAI-SearchBot and GPTBot, PerplexityBot and Perplexity-User, Google-Extended, and Bing's ai-training and ai-grounding preferences.
- Measurement exists, but it is partial. Bing Webmaster Tools reports AI citations and grounding queries, Google counts AI feature traffic inside Search Console's Web search type, and ChatGPT tags referrals with utm_source=chatgpt.com.
- Content quality guidance is unchanged. Google's helpful content guidance (people-first content, clear sourcing, Who, How and Why) is the documented basis, and automation used mainly to manipulate rankings breaches its spam policies.
What is AEO, in plain terms?
A classic search engine returns a list of links and the user chooses one. An answer engine runs searches on the user's behalf, reads the results with a language model and writes an answer with a few cited sources. AEO asks a narrower question than classic search engine optimisation: can the engine reach this page, is it allowed to use it, and does the page state its facts clearly enough to be quoted and linked? The AI Answer Engines Hub explains how the main engines retrieve and cite sources.
| Term | Focus | Typical success signal | Where defined |
|---|---|---|---|
| AEO (answer engine optimisation) | Being selected and cited in AI answers | Citations and referral clicks from answer engines | Industry term; no provider definition |
| GEO (generative engine optimisation) | Visibility of a source inside generative engine responses | Visibility metrics defined in the research | Academic paper, Aggarwal et al., KDD 2024 |
| SEO (search engine optimisation) | Crawling, indexing and ranking in classic results | Rankings, impressions and clicks | Google Search Central documents its own guidance |
As of 2026-11-15. The terms overlap heavily and are often used interchangeably. The GEO paper, first submitted to arXiv on 16 November 2023 and accepted to KDD 2024, introduced a benchmark called GEO-bench and reported that its methods could raise visibility in generative engine responses by up to 40%, with effectiveness varying by domain. It measured research prototypes, not a specific commercial engine. See What Is Generative Engine Optimisation (GEO)? and AEO vs GEO vs SEO.
How do answer engines choose sources?
No provider publishes a full formula for which pages get cited. What the documentation does describe is a pipeline with distinct gates, and AEO work maps to each gate.
- Crawl access: the provider's search crawler must be allowed to fetch the page (Googlebot, Bingbot, OAI-SearchBot, PerplexityBot).
- Index eligibility: the page must be in the engine's index. Google requires indexing plus snippet eligibility for AI features.
- Retrieval: the engine searches its index. Google says AI Overviews and AI Mode may use query fan-out, running several related searches across subtopics and data sources, so a page can be retrieved for a sub-question the user never typed.
- Grounding: selected passages are passed to the model. Microsoft defines AI grounding as using content as input to help inform AI-generated responses, and says this is separate from indexing and ranking for traditional results.
- Citation: the answer links the sources it used, inline or in a sources panel, depending on the engine.
The practical consequence: a page blocked at gate 1 or 2 cannot be cited however well written it is, and a page that passes the gates competes on how clearly it answers each sub-question.
Which signals do providers actually document?
| Provider | Stated requirement to appear in AI answers | Control to stay out of answers | Separate training control |
|---|---|---|---|
| Indexed and eligible to show with a snippet; "no additional requirements" for AI Overviews or AI Mode | nosnippet, data-nosnippet, max-snippet or noindex; Googlebot rules in robots.txt | Google-Extended, which limits AI training and grounding in some of Google's other systems | |
| Microsoft (Bing) | Normal crawl and indexing requirements; grounding preference Allow is the default | Bingbot meta tag or X-Robots-Tag: ai-grounding:disallow or ai-grounding:disallowextended (honoured from 1 February 2027) | robots.txt token __usage_ai-training, or page-level ai-training:disallow |
| OpenAI | "Any public website can appear in ChatGPT search"; do not block OAI-SearchBot | Block OAI-SearchBot (link and title may still appear as navigational links), or noindex on a crawlable page | GPTBot; blocking it signals content should not be used for training |
| Perplexity | Allow PerplexityBot so the site can appear in search results | Block PerplexityBot in robots.txt; Perplexity-User generally ignores robots.txt because a user requested the fetch | Perplexity states PerplexityBot is not used to crawl content for AI foundation models |
As of 2026-11-15. OpenAI says a robots.txt change for OAI-SearchBot takes about 24 hours to take effect, and that if both OAI-SearchBot and GPTBot are allowed it may use one crawl for both purposes. Microsoft warns that choosing Disallow or DisallowExtended for grounding will significantly reduce visibility in AI-generated experiences, and that DisallowExtended limits grounding to Microsoft-operated experiences. Full control syntax for every provider is in the AI Crawler and Grounding Controls Tracker.
Microsoft's dates to diary
| Date | What happens |
|---|---|
| 15 December 2026 | Temporary grounding preferences open in Bing Webmaster Tools for sites that use NoArchive or NoCache |
| 1 February 2027 | New training and grounding preferences take effect as Microsoft crawls |
| 1 March 2027 | Deadline to publish settings and record temporary grounding preferences |
| 30 September 2027 | Temporary grounding preferences expire |
As announced by Microsoft on 30 October 2026. Microsoft notes there is no site-wide grounding setting in robots.txt; grounding is set per page through the meta tag or header.
What makes content citable?
Providers do not publish a separate content standard for answer engines. Google's documented basis is its existing guidance on helpful, reliable, people-first content, and its AI features page says standard SEO practice applies.
- Original value: Google's self-assessment asks whether content gives original information, research or analysis, covers the topic thoroughly and adds value beyond other results.
- Clear sourcing and authorship: Google's "Who" question asks whether the author is clear, with bylines linking to background. Trust is described as the most important part of E-E-A-T, which Google says is not itself a ranking factor.
- How it was made: Google suggests disclosing automated or AI-assisted production where readers would reasonably expect it, and calls large volumes of AI text produced without manual oversight low effort.
- Structured data that matches the page: Google recommends JSON-LD, says markup must describe content visible to users, and states markup does not guarantee rich results. For AI features it says no special schema.org markup is needed.
- Answerable passages: because retrieval can fan out into sub-questions, pages that state definitions, figures and steps plainly under clear headings give the engine passages it can quote. This is a reasonable inference from the documented pipeline, not a provider rule.
What does AEO not include?
Several claims circulate in AEO marketing without support in provider documentation. Treat them with caution and ask for the official source.
| Claim | What the official documentation says |
|---|---|
| A special file or markup is needed for AI answers | Google: no special optimisations and no special schema.org structured data needed for AI Overviews or AI Mode |
| Blocking training crawlers removes you from AI search | OpenAI and Microsoft document training and search or grounding as independent settings |
| robots.txt stops every AI fetch | OpenAI says robots.txt rules may not apply to ChatGPT-User; Perplexity says Perplexity-User generally ignores robots.txt |
| Mass-produced AI content scales visibility | Google says automation used mainly to manipulate rankings violates its spam policies |
| A guaranteed citation uplift | No provider publishes a citation formula; the 40% figure comes from a research benchmark, not a production engine |
As of 2026-11-15. Attempts to plant content so that models repeat it are a known abuse pattern, covered in LLM Grooming and AI Search Poisoning.
How do you measure AEO?
| Tool or signal | Provider | What it shows | Limits stated |
|---|---|---|---|
| AI Performance report, Citations and Pages tab | Microsoft (Bing Webmaster Tools) | Grounding queries where the site appears and the pages cited for each | Surfaces covered not listed in the October 2026 post |
| Topics Portfolio (beta, 13 October 2026) | Microsoft (Bing Webmaster Tools) | Grounding queries clustered into topics, with topic size, sites cited and opportunity queries | Directional numbers, ranges, AI-generated topic labels; hidden when data is thin |
| Search Console Performance report, Web search type | Traffic from AI features, included in overall search traffic | Not broken out from other web search traffic in the documentation fetched | |
| Referral parameter utm_source=chatgpt.com | OpenAI | Clicks from ChatGPT search, visible in web analytics | Requires OAI-SearchBot to be allowed |
| Server logs by user agent | All providers | Crawler visits from Googlebot, Bingbot, OAI-SearchBot, PerplexityBot and user-initiated fetchers | Verify against the IP lists each provider publishes |
As of 2026-11-15. Useful metrics: share of priority pages that are indexed and snippet-eligible; cited pages and grounding queries per month in Bing; referral sessions from answer engines; crawler hits from verified provider IP ranges; and the number of pages with accidental blocks. Third-party citation tools are listed, without endorsement, in the AI Search Citation Reporting Tracker.
How to start: a phased approach
- Decide policy first: for each provider, decide whether you want to appear in AI answers, be used for training, and be used for grounding. Record the decision.
- Audit access: check robots.txt, meta robots and X-Robots-Tag on priority pages for each search crawler named above. Confirm indexing with URL Inspection in Search Console and Bing Webmaster Tools.
- Fix eligibility gaps: remove unintended noindex or nosnippet values, and make sure structured data matches visible text.
- Improve answerability: review priority pages against Google's helpful content questions; add clear authorship, sources and plain statements of key facts.
- Set up measurement: enable Bing AI Performance, segment utm_source=chatgpt.com in analytics and baseline crawler hits from logs.
- Test and roll back safely: change controls on a small set of pages first, wait for recrawl (OpenAI states about 24 hours for OAI-SearchBot), compare citations and referrals, then extend or revert.
Common failure patterns
- One blanket block: blocking every AI user agent to stop training also removes the site from AI search answers.
- Noindex on an uncrawlable page: OpenAI notes its crawler must be able to crawl a page to read the noindex tag.
- Relying on NoArchive or NoCache for Bing: Microsoft says these stop counting as a training or grounding preference after 1 March 2027.
- Markup that does not match the page: Google's guidelines require structured data to describe visible content.
- Chasing volume: bulk AI-written pages without oversight run into Google's spam policies and quality guidance.
Where identity and security fit
AEO decisions are also access decisions about automated clients. Crawler user-agent strings can be spoofed, so verify claimed crawlers against the IP lists that OpenAI and Perplexity publish before allowing or rate-limiting them, and keep crawler rules under change control. See AI Crawler and Scraper Controls.
Practitioner checklist
- Write down, per provider, whether you allow search inclusion, training and grounding.
- Confirm priority pages are indexed and snippet-eligible in Google Search.
- Allow OAI-SearchBot and PerplexityBot if you want ChatGPT search and Perplexity citations; set GPTBot separately.
- Plan Bing ai-training and ai-grounding preferences before 1 February 2027 and record temporary preferences by 1 March 2027 if you use NoArchive or NoCache.
- Turn on Bing Webmaster Tools AI Performance and track cited pages monthly.
- Segment utm_source=chatgpt.com referrals in analytics.
- Verify crawler traffic against provider IP lists rather than user-agent strings alone.
- Ask any AEO supplier for the official provider source behind each recommendation.
Standards and references
- Google Search Central: AI features and your website
- Google Search Central: Creating helpful, reliable, people-first content
- Google Search Central: Introduction to structured data markup
- Microsoft Bing Blogs: Clearer preferences for training and grounding
- Microsoft Bing Webmaster Blog: Topics Portfolio beta in AI Performance
- OpenAI: Overview of OpenAI crawlers
- OpenAI Help Center: Publishers and developers FAQ
- Perplexity: Perplexity crawlers
- arXiv: GEO: Generative Engine Optimization (Aggarwal et al.)
Related NHI Mgmt Group resources: AI Answer Engines Hub · AI Search Engines Compared · Google AI Mode Explained · ChatGPT Search Explained · Perplexity AI Explained