We Asked ChatGPT, Claude, Gemini and Perplexity the Same 24 AI SEO Questions

The four AI platforms did not treat provider recommendations and supporting sources the same way. A brand could appear in an answer without its own website being cited.

The sources behind those recommendations also had a clear commercial tilt: agencies and service providers accounted for the largest share in these runs.

The results below describe this study's four documented runs, not how every platform will behave for every user or query.

The first six prompts all asked some version of the same commercial question:

Which agency should I hire?

For every brand surfaced in those answers, we checked whether the company's own domain appeared in the same answer's visible source list. The gap was substantial.

A company could appear in the recommendation even when its own site was not among the visible sources, which changes what a useful visibility report needs to tell you.

If a company appears in an AI answer, there is a real difference between:

• the company's website supplying the source;
• another publisher supplying the source;
• the company being mentioned without its domain appearing at all;
• the company being discussed but not actually recommended.

Those are different outcomes.

They should not be collapsed into one “visibility” number before the underlying evidence is preserved.

For the six provider-selection questions, being recommended did not mean a brand's own website was being used to support the answer. The share of answers citing the brand's own site varied sharply across the four runs.

The source mix changed when the question moved closer to a purchase.

The first six prompts asked which agencies or providers could help a company improve its visibility in AI-generated answers. Agency roundups appeared frequently. So did software companies, provider comparisons, commercial guides and practitioner content. Gemini's visible citations in this run were concentrated entirely in those six provider prompts. Claude's citations for those provider-selection questions also came entirely from commercial, vendor or practitioner-owned sources. ChatGPT and Perplexity showed broader source sets, but the buying questions still leaned heavily toward organizations participating commercially in the category.

The pattern is useful for two reasons. First, it explains why strong first-party content alone may not be enough to influence provider discovery. Second, references from other credible sites matter more, but they also need context. There is a meaningful difference between earning a mention in a respected publication and buying a spot in an agency ranking built to sell placements. We would not treat those as equally credible references.

The technical and research questions looked different from the provider-selection questions. ChatGPT drew from a mix of search-industry publications, practitioner research, vendor material, academic sources and official documentation.

Perplexity surfaced a much broader set of sources. Claude's visible citations leaned more heavily toward practitioners and web publishers, while Gemini did not expose visible citations for prompts 7–24 in this run.

The point is not to decide which mix looks more sophisticated. Different questions pulled from different parts of the web, and the mix also varied by run.

A question about which agency to hire did not draw from the same part of the web as a question about crawler access, measurement methodology or experiment credibility. What someone asks is part of the visibility picture, not just which platform you check.

Making your own site easier to understand, crawl and cite still matters. The study does not argue otherwise. It shows why that work is only part of the job.

Brands in these agency-selection answers were often surfaced without their own websites being cited. Credible references from other sites are part of the visibility picture too.

For us, the practical test is straightforward.

For a business, the practical takeaway is simple: measure each AI platform separately, and track recommendations and citations as different outcomes. Then look for the gaps: where competitors appear and you do not, where your brand is mentioned without your site being cited, and which credible outside sources keep shaping the answers in your category.

The goal is not to chase every citation. It is to understand which parts of your search visibility you can improve, make the changes that are actually in your control, and then measure whether the picture changes.

• We ran one documented benchmark per engine.
• The four environments did not use identical search, source-selection or personalization conditions.
• The results therefore do not describe every ChatGPT, Claude, Gemini or Perplexity user.
• We cannot show that a particular citation caused a recommendation.
• Citation frequency is not a quality score.
• A larger visible source count does not make one system better.
• A repeat test may produce a different mix of sources.
• The ChatGPT benchmark also predates the standardized run-details block used for the other three files.

For those reasons, we describe this as a four-engine observational study rather than a controlled comparison of the underlying models.

The claims are deliberately narrower than the dataset.

You should not need to accept our interpretation in order to inspect the study.

The public dataset contains the records used to build the findings.

The answer can look settled even when the evidence underneath it is not.

In these four runs, the same 24 questions produced 211 cited domains. Most appeared on only one platform. Brands were frequently recommended without their own domains being cited. Agency and service-provider websites made up most of the source owners behind the provider-selection answers. Technical and research questions pulled from a different mix again.

None of that fits comfortably inside one visibility score. We don't think it should.

• Preserve the question.
• Preserve the exact source.
• Preserve where the result appeared.
• Keep the recommendation separate from the citation.
• Then look for what changes.

You end up with evidence you can investigate, instead of a number you can only watch.