Answer Engines, and Whose Index Each One Uses
Answer engines are AI search products that generate responses from retrieved web sources, usually by querying existing search indexes rather than crawling the web at question time. Google AI experiences depend on Google’s index, Copilot on Bing, while ChatGPT Search and Perplexity combine their own systems with external retrieval sources.
The reason this matters is that AI answers are not produced from an omniscient copy of the web. Before a model can cite, summarise, or rely on a page, some retrieval system must have found that page, been allowed to fetch it, parsed it successfully, and chosen to keep it. If your content is missing from the relevant index, the answer engine may still answer the user, but it will build that answer from other material.
Concretely, an answer engine receives a query, rewrites or expands it, asks one or more retrieval systems for candidate documents, then passes selected snippets or pages into a language model. The model uses that evidence to compose a response and may attach links or citations. Google’s AI search features use Google’s search infrastructure. Copilot uses Bing. ChatGPT Search uses OpenAI search infrastructure plus partners and external providers. Perplexity mixes its own crawl with outside search sources.
The trade-off is dependency and opacity. A product can look like a new search surface while still inheriting the coverage, crawl rules, freshness, and ranking biases of another index. The honest answer to “which AI search should I optimise for?” is: it depends on which index feeds the product your audience uses. Blocking a crawler, serving broken HTML, or being judged low-value upstream can remove you from several downstream answer experiences at once.
Engineers meet this in practice when debugging AI visibility, configuring robots rules, analysing server logs, or explaining why a site is cited in one product but invisible in another. Treat the stack as a dependency chain: product, retrieval provider, crawler access, index inclusion, ranking, then citation. A common misunderstanding is that publication alone is enough. It is not. Discoverability and inclusion in the right backing index are prerequisites.
Common questions
- Which index backs the main AI answer products?
- Google AI Overviews and AI Mode are served from Google’s own search systems. Microsoft Copilot is tied to Bing. ChatGPT Search uses OpenAI’s search infrastructure along with external providers and content partnerships, with Bing historically significant for web retrieval. Perplexity uses its own crawling and indexing as well as outside search sources.
- Why can absence from one index affect more than one AI product?
- Because answer products often share retrieval foundations. If a product depends on Bing, directly or through a provider relationship, a page missing from Bing may be unavailable to every downstream experience that asks Bing-like retrieval for evidence. The model can still write an answer, but it will not be grounded in your page.
- Does an AI model know my site just because it is publicly online?
- No. Public availability is only the starting condition. A crawler must be able to reach the page, your robots policy must allow the relevant bot, the content must be parseable, and the index must keep it. If any of those fail, an answer engine may never retrieve the page for grounding or citation.