An AI Citation is Not Proof. It is the Start of an Audit

How to Audit What an AI Source Shows, Supports and Shapes

By Manuel Hürlimann for GaryOwl.com | Published: 24 October 2026 | Version 1.0 | Sources checked to 24 October 2026

Expertise: Digital Authority Engineering | AI citation pipeline diagnostics | Measurement methodology

Time to read: ~35 minutes · ~7,900 words

Series: DAE Mechanics · Terms: DAE Glossary

A citation in an AI search answer shows that a source was displayed. It does not show that the source supports the claim next to it, that it confirms the claim independently of the other sources, that it shaped the answer, or that it is still true when you read it. As proposed here, an AI citation audit records which sources an AI answer showed, with the system, mode and date of the answer, checks claim by claim whether those sources support the claims, flagging claims without a citation, and traces how far the sources share an origin. It also dates each source and the audit itself, and tests influence only in a separate controlled setting, where one is possible. This article is a synthesis of published research. A citation is not proof. It is the start of an audit.

In short

  1. What you can do: a five-phase audit, ten questions for any study, and a checklist for visibility reports.
  2. What research knows: many citations do not fully support their sentence (in a 2026 audit of Google AI Overviews, 11% of claims were not backed by the cited pages), one answer says little about the next, several sources can share one origin, and experiments show that a correct citation and the influence of the source can come apart; how often this happens in everyday use has not been measured.
  3. Evidence with a date: sources, systems and the measuring tool change, as the week of 8 October 2026 showed for ChatGPT.

Contents: What a citation shows · The four questions · Worked example · Audit protocol · Visibility · Support · Independence · Influence · Time · Research and guidance · Reading a study · What to do · Sources · Evidence ledger

Labels: [Tier A] peer-reviewed research; [Tier A*] preprint; [Tier C] independent review of many studies; [Tier D] study by an industry or professional body; [Tier E] vendor, platform, agency, industry body or trade press publication. Full legend at the end.


What does a visible citation actually show?

Two things are easily confused. Being credited means the reader learns who produced the information, through a link or a name. Being correctly cited means the cited source actually supports the claim it is attached to. A source can be credited prominently without supporting the claim, or carry the whole answer without being credited. This article is about the second.

Peer-reviewed audits show the gap between a shown source and a supported claim at scale. For Google AI Overviews, a team at Washington University in St. Louis issued 55,393 trending queries over 40 days in spring 2026, received 7,583 overviews and split the 7,491 they could evaluate into 98,020 single claims, each checked against the cited pages their pipeline could retrieve: 11.0% of the claims were not backed by those pages (6.98% absent from the pages, 2.66% contradicted, 1.39% ambiguous), which is not a share of false claims; only 41.9% of the overviews were fully backed (Xu, Iqbal and Montgomery, 2026) [Tier A]. Both shares, 11.0% and 41.9%, depend on one extraction and retrieval method, with paywalled pages only partly captured, short-form social media excluded and some pages not crawled. The backed 89% include 4.4% supported only in broader or vaguer terms, and under the most generous assumption about uncrawled pages, the authors put the share not backed at about 5.3%. The 11.0%, the 41.9% and the 5.3% cover only the claims the automatic splitting extracted; in the authors’ check of 100 overviews it captured 83% of the single statements in the text, so by our estimate about one statement in six was never checked. Across nine generative search engines and deep research agents, the DeepTRACE audit found that the share of citations whose source actually supported the statement ranged from about 40% to 80%, in results the authors report as of 27 August 2025 (Venkit et al., 2026) [Tier A]. That range is the one in the paper’s summary; its tables differ in detail, and for one system the text and the table disagree (see the evidence ledger). Both studies count single claims, as this article does.

Across languages, a study coordinated by the European Broadcasting Union and led by the BBC had journalists at 22 public service media organisations in 18 countries and 14 languages judge more than 3,000 answers from ChatGPT, Copilot, Gemini and Perplexity to news questions, collected in late May and early June 2025: 31% had serious sourcing problems, which the broadcasters call missing, misleading or incorrect attributions, among them information the cited source does not support (EBU and BBC, 2025) [Tier D]. The platforms describe their own citations differently. Google writes that AI Overviews “include links to web content that supports the information presented in the overview” (Google, 2025) [Tier E]; the audits above test that statement from outside. Microsoft’s citation report for site owners, whose sources include Copilot and AI summaries in Bing, states that the average number of cited pages “does not indicate ranking, authority, or the role of any page within an individual answer” (Microsoft Bing, 2026) [Tier E].


Which four questions sit behind every AI citation?

This article splits every citation into four questions: visibility (was the source shown?), support (does it carry the claim?), independence (how many original sources are there?) and influence (would the answer change without it?). Each is a test of its own and needs its own kind of evidence. Time is not a fifth question but runs across all four: every answer belongs to one system, one mode (for example a fast or a reasoning mode) and one date.

Four questions behind every AI citationFour separate questions, each with its own evidence: visibility, support, independence and influence. Time runs across all four.1 VisibilityWas the source shown?Evidence: repeated observation2 SupportDoes it carry the claim?Evidence: claim-by-claim check3 IndependenceHow many original sources?Evidence: tracing each origin4 InfluenceWould the answer change?Evidence: controlled testTime runs across all fourone system, one mode, one date
Four separate tests, not a ladder: passing one does not imply passing the next.

Each question supports some statements and not others. The figures below are illustrative.

QuestionA statement it supportsA statement it does not support
Visibility: was the source shown?“Cited in 34 of 100 answers, one to each of 100 test questions (95% interval 25% to 44%), ChatGPT, one month”“Influenced 34% of answers”
Support: does it carry the exact claim it is attached to?“78% of cited claims fully supported”“78% of our content shaped the answer”
Independence: does it confirm the claim independently?“Twelve websites traced back to three original sources”“Twelve sources confirmed it independently”
Influence: would the answer differ without it?“Removing X changed 4 of 15 claims in this fixed test”“X controls how AI answers this topic”
Time: for when does it hold?“Valid as of 14 October, for this system and mode”“This is how ChatGPT answers”

The four questions evaluate the link between citations and claims. They do not replace a check of whether a claim is true or a source is credible.

The four questions are close to how current research describes the problem. A critical survey of 45 studies on generative engine optimisation proposes a seven-part “visibility vector” for a source, from discoverability through citation, prominence, absorption and fidelity to behaviour, and its table of metrics has a column headed “What it does not establish”: a citation metric does not establish “exact support or substantive contribution” (Martinez, 2026) [Tier C]. Our visibility and support match the vector’s citation, prominence and fidelity; its absorption measures how much content is taken up, not, like our influence, whether the answer causally depends on the source; independence has no place in the vector.


A worked example: one answer, four questions

An AI answer to “Which European bank has the strongest capital position?” contains five claims and three citation markers, pointing to two documents: [1] a regulator’s report, and [2] and [3] two passages of one trade article. The example is constructed for illustration, as if from one question in English put once to one AI search assistant, logged out, in default mode, with snapshots of both documents and their times; a real audit would repeat the run.

Worked through below, the audit gives this result: three citations are shown, but only one of five claims is fully supported; the capital claims rest on one original source; influence was not tested, since the interface gives no control over which pages are found; and whether one figure was still current is not established. The audit supports a statement about this answer on this day, not about how often this happens or which source shaped the answer.

ClaimMarkerWhat the source saysJudgement
The bank leads its peer group in core capital[1]Peer comparison table in the reportFully supported
Its core capital ratio is 17.2%[1]17.2%, but for an earlier quarter than the answer impliesPartly supported
It is the safest bank in Europe[2]The article discusses capital, not overall riskUnsupported
Regulators praised its buffer strategynonenoneNo citation
It has had no state support since 2010[3]The article reports support in 2012Contradicted

Rules like these must be fixed before the check. Here, the claim names no period, so a matching figure from an earlier quarter counts as partly supported; had it named a quarter and the source another, it would count as contradicted.

Support. A visibility dashboard would have counted three citations; a claim-by-claim check finds one fully supported claim out of five. Citation recall asks whether the cited passages together fully back a sentence and counts a sentence without citation as unsupported (Gao et al., 2023) [Tier A]. Here that is one in five or 20%. Gao et al. count sentences; this article applies the rule to single claims, so its figures are not directly comparable with sentence-based ones. Because the rule folds two questions into one figure, we report two shares beside it: the support rate among cited claims, one of four or 25%, and the share of claims that carry a citation, four of five or 80%. Only together do they show whether an answer cites little but cites well, or the reverse; all three belong next to the five judgements. DeepTRACE’s citation accuracy and thoroughness are related but different measures (Venkit et al., 2026) [Tier A]; see the evidence ledger.

Independence. The capital claims cite two documents, but the trade article takes its figures from the regulator’s report: two documents, one origin. The origin of the claim about state support cannot be traced and is recorded as unknown, not as independent.

Influence. Not tested: the interface gives no control over which pages are found, so nothing can be said about which source shaped the answer.

Time. The 17.2% ratio is supported for an earlier quarter; whether it was still current when the answer was given is not established.

What the audit supports: “In this answer, on this day, one of four cited claims was fully supported, and the capital claims rest on one original source.” What it does not support: any statement about how often this happens, or about which source shaped the answer.


How do you run a citation audit?

A citation audit has five phases: define the test set, keep every answer as raw data, check support, trace origins, and test influence separately. The audit record, the independence grading (I0 to I5, see the section on independence) and the two shares reported next to citation recall are our proposals, none of them validated (see the box at the end of this section); the sections after these five phases give the research behind them.

  1. Define the test set before running anything: questions, language, market, time window, system and mode, how answers are collected (logged in or out, web or programming interface, country, any data supplier), how often each question is repeated, which answers count, and whether results will describe only this set of questions or the wider topic they are drawn from. Record a starting point before changing any content, and audit each engine separately: the method transfers, the numbers do not. Research also asks for several wordings of each question, because results depend on the wording; parallel runs from a second profile or location at the same time, the classic way to separate noise and personalisation from a change in the system (Hannák et al., 2013) [Tier A]; and, where an engine offers both, paired runs through the consumer interface and the programming interface, which can return different sources.
  2. Keep every answer as raw data: question, run, time, full answer text, all visible citations, and a snapshot of each cited page. Note how the citations were captured, and check that capture against a second method on a sample before relying on it; links in the text, in the sources panel and in product panels can be tagged differently.
  3. Check support: split the answer into single claims under a written rule, match each to its citation and passage, judge it, and flag claims without a citation. If a cited page cannot be retrieved, mark the claim as not assessable, not as unsupported. Automated judging can help; claims that matter need a person.
  4. Trace origins: find each source’s origin, author, data and date, mark shared origins, and assign an independence grade (see the section on independence).
  5. Test influence separately, in a controlled setting; decide in advance which change in the answer counts (a different fact or recommendation, not a new wording or only a different citation), and limit conclusions to that setting.

The audit record holds one row per pair of claim and citation, plus one row for each uncited claim, flagged “no citation”.

FieldContent
ClaimID and exact wording
CitationCited address, snapshot, and the passage used
SupportJudgement and a short reason
OriginOrigin of publication, data and method, shared authors or funding, recorded separately; or unknown
CredibilityHow reliable author, method and data are for this specific claim
AccuracyHow the claim relates to the best available evidence, if checked; support is not truth
TimeSource date, answer time with system and mode, audit date
InfluenceTest and result, or not tested

For citation recall and the two shares beside it, results go back to the level of the claim: a claim counts once, however many citations it has, and a written rule decides whether several partly supporting citations together count as full support. Before adding up results across many answers, decide in writing how answers without citations, claims that cannot be assessed, and questions of different importance are counted. Negative or null findings belong in the report too: a claim that no citation supports, or a page that no answer cites, is a finding.

Our proposals, not standards

The audit record, the independence grading I0 to I5 and the two shares reported next to citation recall are proposals, none of them validated. Before results can be compared across audits, five things need to be fixed and tested: claim splitting, counting of partly supported claims, handling of unclear or conflicting sources, agreement between independent reviewers (inter-rater agreement), and weighting by importance. Until several independent reviewers have graded the same sources and reported their agreement, the grading is a checklist, not a measurement.


Visibility: was the source shown?

Citation visibility is the share of answers to a set of test questions that cite a source. Citation share is the source’s part of all citations shown. Both can be counted for a whole website or for a single page, and the two differ: a website can be cited often while a given page is never cited. Two things are easy to miss when reading a visibility figure: which base it is a share of (all queries, those that produced a generative answer, or those where the system searched) and its range, because answers vary from run to run; both are explained below.

Being linked and being named are different kinds of visibility. A study by Semrush with Kevin Indig counted 3,981 appearances of websites in answers from four AI search engines to 115 questions in 14 countries: in 61.7% of them, a website was linked as a source but its brand was not named, or about 82% of the linked appearances by our calculation (Loktionova, 2026) [Tier E]. It is a market observation by a vendor whose own tool collected the data. The split also differs by engine: ChatGPT linked sources in 87% of cases and named brands in 20.7%, while Gemini did the opposite, with 21.4% and 83.7%. The study’s bases differ between the engines and none is given for the 87%, so these figures are indicative only. A figure that averages both engines describes neither.

For a source to be cited through live web retrieval, a generative answer has to appear and the system has to search and find the page. Whether every displayed citation rests on such a search is not settled; the evidence points both ways (see the influence section). In the Washington University audit, Google showed an AI Overview for 13.7% of trending queries and for 64.7% of those phrased as a question (Xu, Iqbal and Montgomery, 2026) [Tier A]. In an analysis of about 14,000 logged conversations with search-enabled models, 15.6% of answers skipped the web search entirely although search was switched on (Strauss et al., 2026) [Tier A]. A visibility figure therefore needs its base stated: of all queries, of those that produced a generative answer, or of those that searched.

Because answers vary from run to run, a single visibility figure needs a range. If eighteen of fifty answers cite a source, that is 36% of those fifty answers; as an estimate of what the system does in general, it carries a range of roughly 24% to 50% (Wilson, 1927) [Tier A]. That range assumes fifty independent answers. If they come from five questions asked ten times each and answers to the same question resemble each other, the range is too narrow, by an amount that depends on how alike they are and on whether the aim is to describe these five questions or the wider topic. A report should then give the number of questions and days next to the number of answers, report results per question, and compute the range with a method that accounts for this grouping, such as a cluster bootstrap over questions (Cameron, Gelbach and Miller, 2008) [Tier A] or clustered standard errors (Miller, 2024) [Tier A*].


Support: does the source carry the claim?

A citation can support a claim fully, support it in part, fail to address it, or contradict it, and that can only be judged claim by claim. Research on attribution, the link between a statement and its source, uses two metrics: citation recall, used in the example above, and citation precision, whether each citation contributes relevant support; a redundant citation that also supports the sentence is allowed (Gao et al., 2023).

Nelson F. Liu and colleagues checked four generative search engines in 2023. Only 51.5% of sentences in their answers were fully supported by their citations, and 74.5% of citations supported the sentence they were attached to (Liu, Zhang and Liang, 2023) [Tier A]. These are averages for the systems of that time, not error rates for today’s systems.

Automated checking helps but does not settle the question. In a large evaluation, GPT-4o and human assessors gave the same support label in 56% of cases when the humans judged on their own, and in 72% when they corrected the model’s labels (Thakur et al., 2025) [Tier A]. Both figures measure agreement, not accuracy. In the DeepTRACE audit, the automated judge agreed with human raters on factual support at a correlation of 0.62, which its authors call moderate (Venkit et al., 2026) [Tier A].

How an answer is split into claims decides what gets counted. Research on factual precision splits long answers into atomic facts, each checked on its own (Min et al., 2023) [Tier A], and the Washington University audit splits AI Overviews into atomic claims the same way (Xu, Iqbal and Montgomery, 2026); a written segmentation rule should follow that practice.


Independence: how many original sources are there?

Five websites repeating a claim are not automatically five confirmations. An original study feeds a press release, two trade outlets and a newsletter rewrite it, and an answer cites the three rewrites as separate evidence. The answer shows three websites; the evidence has one origin. Independence is judged for one claim at a time: two pages can share the origin of one figure and be independent for another. An audit therefore counts origins, not websites. In our proposal, the audit traces each cited page, claim by claim, to the publication, data and method the page descends from, notes institutional ties next to the grade without letting them change it, records an origin it cannot trace as unknown, never as independent, and grades the dependence in steps from I0 (duplicate) to I5 (independent confirmation with different methods), set out below.

In science, Steven A. Greenberg traced a network of 242 papers and showed how citation bias, amplification and invention can create “information cascades resulting in unfounded authority of claims” (Greenberg, 2009) [Tier A]. For AI answers this is a hypothesis: plausible by analogy, but the studies reviewed here do not show how often it happens.

In data integration, detecting which sources copy from which is a known problem, called source dependence, with its own methods (Dong, Berti-Equille and Srivastava, 2009) [Tier A]. DeepTRACE measures “source necessity” (which sources are needed, not whether they share an origin), the smallest share of the listed sources needed to support all statements of an answer: for the generative search engines it audited, 50% to 69% were needed, so, by DeepTRACE’s measure, roughly a third to a half added nothing the others did not already support (results as of 27 August 2025) (Venkit et al., 2026) [Tier A]. An audit of four AI search engines on 712 real questions found that about 16% of the cited sources were themselves AI-generated (Allaham and Diakopoulos, 2026) [Tier A]; AI authorship is a separate attribute, not a measure of dependence, and an audit records it as its own field. None of these studies traces a cited page back to the publication, data or method it descends from. The nearest published work checks, for 256 search trajectories of Tencent’s search assistant Yuanbao, whether the cited support goes back to at least two independent evidence roots: a yes or no check, by authors at Tencent and two Chinese universities (Zhao et al., 2026) [Tier A*]. We found no published work that records publication, data, method and institutional ties as separate fields or grades independence in steps; our search of 18 October 2026 (see the method note) covered independence only and was not systematic, and not finding something is not proof that it does not exist. An audit can establish a shared origin, not whether anyone built it on purpose.

Our own working categories of source dependence, ordered by how much of the evidence is shared, not by the quality of the studies:

GradeMeaningExample
I0DuplicateThe same page under two addresses
I1Direct reproductionA newsletter reprints a press release
I2Shared originTwo trade articles both rewrite one study
I3Shared dataTwo independently written analyses of one public dataset
I4Independent evidenceTwo studies with their own data on the same question
I5Independent confirmation with different methodsA survey and a controlled experiment that agree

The grade is a shorthand, not a ranking of quality: reproducing a study with the same method and confirming it with another method test different things, and two different methods can still share the same blind spots. The record behind it, published next to any grade, holds four fields: origin of publication (own or reproduction), of data (shared or separate) and of method (same or different), and institutional ties (shared authors, organisations or funding). The grade follows the most dependent of the first three: the same page under two addresses is I0; a text that only restates another, word for word or in its own words, is at most I1; two texts restating a third share an origin and are at most I2; shared data caps the grade at I3; the same method with separate data allows I4 but not I5; I5 needs separate data and different methods. Institutional ties bear on interest and credibility, not on whether the evidence is shared, so they do not change the grade and are reported next to it. Two independently published articles on one dataset are therefore I3, not I4; two studies by one team with their own data and the same method are I4, with the shared authors noted. Where an axis is unknown, the grade is a range, from what an unfavourable answer would give to what the known axes allow, for example I2 to I4. An origin that cannot be traced is recorded as unknown, never as independent. Grades are steps, not scores: they are not averaged.


Influence: would the answer change without the source?

A citation can be correct and still not be the reason for the sentence. Influence needs a controlled change to a source, and on a public interface such as ChatGPT an audit of live answers can raise suspicions about influence but not prove it. An unchanged answer does not show that the source was not used, and how often citation and influence come apart in everyday use is not known. Studying systems that answer from retrieved pages (retrieval-augmented generation), Wallat and colleagues call a citation faithful only if the statement actually depends on the cited document (Wallat et al., 2025) [Tier A]. In a laboratory test with one model (Command-R+) on 1,444 questions from a public question set, they appended short statements from the model’s earlier answers to other documents; where such a document replaced one the model had judged relevant but not cited, the model cited it for the same statement in 273 of 476 cases in which the statement came back, or 57%, which is not an estimate for everyday use. The authors read this as the model answering from what it already knew and attaching a matching document afterwards, assuming, as they say still needs checking, that the inserted text does not change how the model works inside.

Influence can only be tested by changing something and watching what happens: removing a source, replacing it with a comparable one, or changing one fact inside it while the question stays the same. Researchers with access to a model’s internals can do this precisely. On a public interface such as ChatGPT, nobody controls which pages the system finds, so an audit of live answers can raise suspicions about influence but not prove it; a before-and-after comparison, with the expected change written down in advance, unchanged comparison pages measured alongside and the normal day-to-day swing known from the period before, supports at most a suspected effect. A report should therefore keep apart observed use, which pages a system fetched and cited in an actual run as far as its logs show, and the measured effect of a source in a described test. The first does not show the second, and an unchanged answer in a test does not show that the source was not used, since another source with the same content may have taken its place. OpenAI’s API documentation for web search draws a related line: its list of sources “returns the complete list of URLs the model consulted”, and “The number of sources is often greater than the number of citations” (OpenAI, undated) [Tier E]. Observed use is incomplete from the side of a website too: if a system answers from an index built in advance, the website’s server logs show less of its use in single conversations. In log files of the enterprise clients of a vendor of log analysis, events by the ChatGPT-User agent fell by 28% from 1 December 2025 to 14 March 2026 compared with the period before, while events by OpenAI’s search crawler, OAI-SearchBot, were 3.5 times as many after the launch of GPT-5 in August 2025 as before (Long, 2026) [Tier E]. The author leaves open whether fewer users or more use of OpenAI’s own index explain the fall, and the post does not say how the bots were verified.

Two levels are easily confused:

LevelQuestionEvidence
Textual overlapHow much of the source’s wording appears in the answer?Observable by comparing texts
Causal influenceWould the statement differ had the source been absent or different?Only a controlled test

Overlap shows that two texts agree, not where the wording came from: it may come from a shared origin or from what the model already knew. Measures that score how much of a source an answer takes over are useful descriptions, but they do not measure cause (Zhang, He and Yao, 2026) [Tier A*].

How often citation and influence come apart in everyday use is not known. What has been measured is a narrower gap, between the pages a system retrieves and the pages it shows. In the logged conversations cited above, eleven models from OpenAI, Perplexity and Google in spring 2025, Gemini and Perplexity’s Sonar typically left about three relevant websites uncited per query; 34% of Gemini answers and 24% of GPT-4o answers were produced without retrieving any online content; 92% of Gemini answers carried no clickable citation (Strauss et al., 2026) [Tier A].

The reverse gap appears in a vendor-side observation of a current system. A French search agency, which sells search consulting, compared the pages cited in 1,249 ChatGPT answers, captured in July 2026 on real accounts and in logged-out sessions, with the search results ChatGPT had received for each, as read from its answer stream: 85% of citations in instant mode and 31% in thinking mode pointed to a URL absent from that list (Resoneo, 2026) [Tier E]. The agency cannot yet tell its two explanations apart: links carried by widgets such as maps and product cards, and brand homepages taken from the model’s memory. A stream field naming the search engine behind each result has not been seen since 21 July 2026, which limits checks on later data. Such citations need not be wrong: a model can cite a fact it already knew with a page that contains it. For Google AI Overviews, a peer-reviewed audit found 52.7% of citations (20,917 of 39,673) for 2,597 queries on health, finance, legal and safety topics pointing outside the up to 100 organic Google results the authors had collected for each query, their own list rather than the set the system fetched (Kakimov et al., 2026) [Tier A]; two of the four authors work for a company that sells detection of AI-written text, which the study uses for other findings, not this one. A university study gives a different picture for ChatGPT’s API (gpt-5.4-nano), with English questions about the US market and the caveat that the API need not behave like the consumer application: in a replay with runs up to 7 October 2026, all 22,315 cited URLs were among the pages ChatGPT’s searches had returned. Brand names were a different matter: in the study, 46% of ChatGPT’s mentions of brands not named in the question had no cited page naming the brand, which the authors read as names from the model’s own knowledge (Chen and Koudas, 2026) [Tier A*]. Systems, access, modes, models and months differ, so these figures cannot be set against each other. Comparing cited pages with a list of search results measures how far the two differ, not causal influence.


Why is every citation evidence with a date?

A source can disappear, be edited or become outdated between publication and answer, and the AI system itself changes. Research outside AI search has shown each of these:

  • The right answer can depend on when you ask. For about 16.5% of questions in a standard question set, the correct answer depends on when, or where, the question is asked (Zhang and Choi, 2021) [Tier A].
  • Outdated sources can mislead a model even when current information is available (Ouyang et al., 2025) [Tier A].
  • A recent date can be faked. Adding a recent date to otherwise unchanged text moved it up in every one of the seven AI ranking systems tested (Fang et al., 2025) [Tier A*]. A recent date is not evidence that a claim is current.
  • Cited pages change. One in five scholarly articles suffered from reference rot: cited web pages no longer showed the cited content (Klein et al., 2014) [Tier A], and three out of four web references led to content that had changed since it was cited (Jones et al., 2016) [Tier A].
  • The system changes. The same named model behaved substantially differently three months apart (Chen, Zaharia and Zou, 2024) [Tier A].

For an audit this means three dates per citation: when the source was published or last changed, when the answer was produced (with system and mode), and when the audit was done. A further step asks whether a supported claim is still current at the time of the answer and at the time of reading. Being current does not make a claim true.

The measuring tool changes too. A data supplier can alter its format, and an interface can change the form of its answers overnight. Keep a log of such changes and split your figures at each break, as the industry framework of the Interactive Advertising Bureau (IAB) also asks, recording the trigger and date of each baseline reset (IAB, 2026) [Tier E]; compare across a break only once the capture and the meaning of the metric are shown to be unchanged. Research on generative engine optimisation lists “treating drift as an object of study” as an open task and asks for versioned questions, snapshots and page content (Martinez, 2026) [Tier C]. The following case shows why.

Case: one week in October 2026, two breaks

On 7 October 2026, OpenAI began rolling out GPT-6 with a new answer format, to paying users first and to the free tiers from the next day (OpenAI, 2026) [Tier E]. Analytics vendors and the trade press, all of which sell or use AI search analytics, then reported two changes. The format, which nobody has called a bug: direct URLs appeared in the answer text in 42% of answers to one million prompts after 8 October, against 8% to 10% before (Southern, 2026) [Tier E], and the new format, which one vendor calls Intelligent UI, made up about a quarter of answers on 9 October and about a third by 14 October, with no figure for earlier days (Blyskal, 2026) [Tier E]. The tag: from 8 October, links in the answer text lost the parameter utm_source=chatgpt.com and most clicks arrived without a referrer, so ChatGPT visits appeared to fall while direct visits rose. A product leader at Adobe stated that OpenAI had confirmed a bug to him on 14 October and that on 15 October tagged clicks were back to roughly 95% to 100% of the same hour a week earlier, the referrer still missing on most clicks (Southern, 2026) [Tier E]; his posts, as quoted by the trade press, read “There’s no tag and no referrer” on 14 October and “Relieved. It was a bug.” on 15 October (Schwartz, 2026) [Tier E]. Profound’s sample showed the tag back in about 99% of cited answers only at 06:30 UTC on 16 October (Blyskal, 2026) [Tier E]. Both trade reports relay statements by this one person and cite nothing from OpenAI; Profound had seen no official word on a fix, and OpenAI’s release notes for 1 to 15 October do not mention link tracking (Southern, 2026).

Report figures from before and after the format change separately until the capture and meaning of the metric are shown to be unchanged. Tag figures from 8 to 15 October are not evidence of lost visits, though the reports cannot exclude a real change in the same days. In analytics, annotate the week and check that the tag has returned. Underneath runs a longer trend: ChatGPT has been reported to cite fewer listicles, articles built as a ranked list, by 30% from December 2025 to January 2026 on a fixed set of questions (Haigler, 2026) [Tier E], and to retrieve fewer after GPT-5.6 in August 2026, where a vendor saw their share of retrieved pages fall from 15.47% to 8.19% (Konitzny, 2026) [Tier E]; no single week explains this.


How does this method compare with current research and industry guidance?

Before publication we checked the method against research on AI search from 2023 to October 2026 and the IAB’s August 2026 framework for visibility measurement: “supported” (research or the framework does or requires it), “extension” (no published equivalent found, so a practical proposal), “to be validated” (not tested by anyone) and “behind” (research has better tools).

Overall: research or the framework supports the method on visibility reported as a distribution and per engine, on judging support claim by claim (in principle), on separating influence from overlap, on the event log with separate series after a break, and on the five-phase protocol, published code and checksums. Recording origins in separate fields and grading independence in steps extend a yes or no check and are still to be validated; the three dates per citation are an extension. The method is behind on statistics for repeated runs, on how claims are split and judges are checked, and on three controls in the protocol: paraphrases, parallel runs, and the consumer interface against the programming interface. Pre-registration of our own measurements is an intention, not yet done.

Part of this methodAgainst research and guidanceNearest published work
Visibility as a distribution, reported per engineSupportedSchulte, Bleeker and Kaufmann, 2026 [Tier A*]; Zhang, He and Yao, 2026 [Tier A*]
Support judged claim by claimSupported in principle; behind on how claims are split and how judges are checkedGao et al., 2023; Min et al., 2023; Thakur et al., 2025; Xu, Iqbal and Montgomery, 2026 [Tier A]
Independence: original sources behind shown onesExtension of a yes or no check; to be validatedZhao et al., 2026 [Tier A*]; Dong, Berti-Equille and Srivastava, 2009; Venkit et al., 2026 (source necessity); Allaham and Diakopoulos, 2026 [Tier A]
Influence separated from overlapSupportedWallat et al., 2025; Strauss et al., 2026 [Tier A]
Time: three dates, event log, splitting at breaksSupported for the event log and separate series after a break; the three dates per citation are an extensionIAB, 2026 [Tier E]; Chen, Zaharia and Zou, 2024 [Tier A]; Martinez, 2026 [Tier C]
Statistics for repeated runsBehindMiller, 2024 [Tier A*]; Cameron, Gelbach and Miller, 2008 [Tier A]
Five-phase audit protocolSupported; behind on paraphrases, parallel controls and interface against programming interfaceMartinez, 2026 [Tier C]; Hannák et al., 2013 [Tier A]
Frozen questions with checksums, event log, published code; pre-registrationSupported for code and hashes; pre-registration of our own measurements is an intention, not yet doneSeveral 2026 studies publish code and hashes; pre-registration before data collection is rare in this field

Where we are behind. Statistics: methods for grouped answers that work well with many groups understate uncertainty with few (Cameron, Gelbach and Miller, 2008) [Tier A]; accounting for grouped questions has widened standard errors in language model evaluations by up to three times (Miller, 2024) [Tier A*]. With few questions, even a model with question and day as random effects can understate uncertainty. An analysis plan fixed before data collection should state the primary outcome and its denominator, how questions and days enter the model and how many are planned, how missing answers and claims that cannot be assessed are counted, which intervals are reported, and how model fit is checked. Judging support: research now expects atomic claims, a sample judged twice by people working independently with the agreement reported, and any automated judge checked against people first.

Where we extend. Recording origins in separate fields and grading independence in steps have no published counterpart that we found beyond a yes or no check in one commercial assistant (Zhao et al., 2026) [Tier A*]. The three dates per citation are a practical addition; whether other audit traditions use them, we have not examined.


How should you read a study about AI citations?

Reports use “citation rate”, “visibility” and “share of voice” for different things: a link shown, a page retrieved, or information used in the answer. The IAB framework proposes shared definitions: a citation is a link or a named reference to the source, and share of voice is a brand’s share of all brand mentions in a defined category (IAB, 2026) [Tier E]. The framework reads citation rate as a sign that a brand is relied on; this article treats that as a question to test. Ten questions later in this section help check any study: what “citation” means, the base of each percentage, how the page was found, repetition, controls, engines, support, transparency, interests, and durability with outcomes.

Two kinds of study answer different questions:

  • Controlled experiments hand candidate pages directly to a model and change one feature at a time. One of the largest, with 252,000 trials across six models and 18 content factors, found that topical relevance and the position of a page in the list handed to the model mattered most for which source was cited first (Vishwakarma, Kumar and Jamidar, 2026) [Tier A]. The study is peer reviewed; its authors work for a software vendor. Such experiments show what happens once a page is in front of the model, not whether a live engine finds it.
  • Repeated observation of live engines measures what people actually see. Because answers vary, it needs many runs a day, pooled over two to four weeks (Schulte, Bleeker and Kaufmann, 2026) [Tier A*]. It shows how often a source appears, not why.

The strongest evidence combines both: an experiment to test a mechanism, then repeated live observation to check that the effect carries over. A live study should also state the country and language of its queries, since engines localise, and note that where an engine replies with a question, a study of single answers captures only the first turn.

Ten questions help with any new study:

  1. Outcome: does “citation” mean a link shown, a page retrieved, or information used?
  2. Base: is the percentage per question, per answer, or per retrieved page?
  3. Discovery: did the engine find the page itself, or was the page handed to the model?
  4. Repetition: how many runs per question, over how many days, and is a range given?
  5. Controls: were the order of sources, the wording of questions and brand familiarity held constant or varied on purpose?
  6. Engines: are results reported per engine, or averaged across engines that work differently?
  7. Support: did the cited source carry the claim it was attached to?
  8. Transparency: are the method, the questions and the raw answers available for inspection?
  9. Interests: does the author sell the service the study evaluates? A commercial interest does not invalidate a finding; it belongs in the reading of it.
  10. Durability and outcomes: was the effect measured again weeks later, and is there evidence of traffic, enquiries or more accurate answers, or only of more citations?

What should a vendor of AI visibility reports disclose about its methodology? The same points the ten questions above ask of a study. At minimum: the set of test questions and its version, with any change between reporting periods shown separately; the engines, modes, country and language; how many runs per question, on which days; the base of every percentage and its range; which of the four questions each figure answers; where support or influence is claimed, how it was checked; and any commercial interest in the result. The IAB framework sets out required disclosures in more detail, at two levels: a minimum for data used to watch trends, and a fuller one for data used in budget or strategy decisions. Where a provider cannot or will not disclose a required item, the framework treats that absence as a signal (IAB, 2026) [Tier E]. Two points above go beyond its list: which of the four questions a figure answers, and how influence was tested.


What should you do with AI visibility reports and metrics?

If you receive visibility reports: check them against the minimum disclosure list in the previous section (the set of test questions and its version, with changes between periods shown separately; engines, modes, country and language; how many runs per question, on which days; the base and range of every percentage; which of the four questions each figure answers; where support or influence is claimed, how it was checked; and any commercial interest in the result), and ask whether the period crosses a change in the system or the interface, such as the week of 8 October 2026.

If you report AI visibility to a board: the metrics that belong in a board report are those that answer the visibility question with their base and range, for example “cited in 34 of 100 answers, one to each of 100 test questions (95% interval 25% to 44%), ChatGPT, one month”, reported per engine and per group of questions. The range describes how answers vary for that set of questions, not whether the questions represent what customers actually ask; with repeated questions, it must account for the grouping of answers by question. Used as a key performance indicator (KPI), such a figure shows that a source was shown under stated conditions. It does not show that the source supported the claims it was attached to, that the evidence was independent, or that the answer would have changed without it; a metric that claims any of these belongs in the report only with its own audit.

If you publish content for AI search: keep dated records of what each answer showed, and put your effort into what an audit can verify: a documented method, data others can inspect, clear dates, and sources that confirm your claims independently.

Honest Limitation

The four questions are a synthesis of published research, not a validated instrument, and the audit record, the independence grading and the two shares reported next to citation recall are our own proposals (see the box “Our proposals, not standards”). Most research on support and influence used fixed test collections, not the answers people see, and the live studies cited here cover a few engines, languages and months. Our own pilot measurement of ChatGPT informed this article but is not reported in it; it appears separately, with its protocol, data and limits.

None of the four questions, nor the date for which an answer holds, is settled by the citation marker alone. A citation is not proof. It is the start of an audit.


Sources

  1. [Tier A] Gao, T., Yen, H., Yu, J. and Chen, D. (2023). “Enabling Large Language Models to Generate Text with Citations.” EMNLP 2023, pp. 6465 to 6488. DOI 10.18653/v1/2023.emnlp-main.398. Citation recall and citation precision. aclanthology.org/2023.emnlp-main.398 (Accessed: 12 October 2026)
  2. [Tier A] Liu, N. F., Zhang, T. and Liang, P. (2023). “Evaluating Verifiability in Generative Search Engines.” Findings of EMNLP 2023, pp. 7001 to 7025. DOI 10.18653/v1/2023.findings-emnlp.467. 51.5% of sentences fully supported; 74.5% of citations supportive. aclanthology.org/2023.findings-emnlp.467 (Accessed: 12 October 2026)
  3. [Tier A] Thakur, N., Pradeep, R., Upadhyay, S., Campos, D., Craswell, N., Soboroff, I., Dang, H. T. and Lin, J. (2025). “Assessing Support for the TREC 2024 RAG Track: A Large-Scale Comparative Study of LLM and Human Evaluations.” SIGIR 2025, pp. 2759 to 2763. DOI 10.1145/3726302.3730165. Agreement 56% from scratch, 72% with post-editing. doi.org/10.1145/3726302.3730165 (Accessed: 12 October 2026)
  4. [Tier A] Wallat, J., Heuss, M., de Rijke, M. and Anand, A. (2025). “Correctness is not Faithfulness in Retrieval Augmented Generation Attributions.” ICTIR 2025, pp. 22 to 32. DOI 10.1145/3731120.3744592. Up to 57% of citations lacking faithfulness under test conditions: one model (Command-R+), short statements from earlier answers appended to other documents; the authors assume the insertion does not change the model’s internal process. doi.org/10.1145/3731120.3744592 (preprint: arxiv.org/abs/2412.18004) (Accessed: 23 October 2026)
  5. [Tier A] Wilson, E. B. (1927). “Probable Inference, the Law of Succession, and Statistical Inference.” Journal of the American Statistical Association 22(158), pp. 209 to 212. doi.org/10.1080/01621459.1927.10502953 (Accessed: 12 October 2026)
  6. [Tier A] Greenberg, S. A. (2009). “How citation distortions create unfounded authority: analysis of a citation network.” BMJ 339:b2680. DOI 10.1136/bmj.b2680. 242 papers, 675 citations, 220,553 supporting citation paths. pmc.ncbi.nlm.nih.gov/articles/PMC2714656 (Accessed: 12 October 2026)
  7. [Tier A] Zhang, M. J. Q. and Choi, E. (2021). “SituatedQA: Incorporating Extra-Linguistic Contexts into QA.” EMNLP 2021, pp. 7371 to 7387. DOI 10.18653/v1/2021.emnlp-main.586. aclanthology.org/2021.emnlp-main.586 (Accessed: 12 October 2026)
  8. [Tier A] Klein, M., Van de Sompel, H., Sanderson, R., Shankar, H., Balakireva, L., Zhou, K. and Tobin, R. (2014). “Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot.” PLOS ONE 9(12): e115253. doi.org/10.1371/journal.pone.0115253 (Accessed: 12 October 2026)
  9. [Tier A] Jones, S. M., Van de Sompel, H., Shankar, H., Klein, M., Tobin, R. and Grover, C. (2016). “Scholarly Context Adrift: Three out of Four URI References Lead to Changed Content.” PLOS ONE 11(12): e0167475. doi.org/10.1371/journal.pone.0167475 (Accessed: 12 October 2026)
  10. [Tier A] Chen, L., Zaharia, M. and Zou, J. (2024). “How Is ChatGPT’s Behavior Changing Over Time?” Harvard Data Science Review 6(2). doi.org/10.1162/99608f92.5317da47 (Accessed: 12 October 2026)
  11. [Tier A] Ouyang, J. et al. (2025). “HoH: A Dynamic Benchmark for Evaluating the Impact of Outdated Information on Retrieval-Augmented Generation.” ACL 2025, pp. 6036 to 6063. arxiv.org/abs/2503.04800 (Accessed: 12 October 2026)
  12. [Tier A] Vishwakarma, R., Kumar, S. and Jamidar, R. (2026). “What Gets Cited: Competitive GEO in AI Answer Engines.” SIGIR 2026. DOI 10.1145/3805712.3808445. 252,000 trials, six models, 18 content factors, two candidate sources per trial. All three authors are employed by Sprinklr. arxiv.org/abs/2605.25517 (Accessed: 12 October 2026)
  13. [Tier A] Xu, H., Iqbal, U. and Montgomery, J. M. (2026). “Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact.” Proceedings of the ACM Internet Measurement Conference (IMC 2026), Karlsruhe, 12 to 16 October 2026. DOI 10.1145/3777912.3839818. 55,393 queries over 40 days (13 March to 21 April 2026); 7,491 overviews; 98,020 atomic claims; 11.0% not backed by the cited pages; 41.9% of overviews fully backed; activation 13.7%, 64.7% for question-form queries. arxiv.org/abs/2605.14021 (Accessed: 17 October 2026)
  14. [Tier A] Venkit, P. N., Laban, P., Zhou, Y., Huang, K.-H., Mao, Y. and Wu, C.-S. (2026). “DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence.” ICLR 2026. Eight dimensions including source necessity, citation accuracy and citation thoroughness; 303 queries, nine systems; citation accuracy about 40% to 80% in the summary, 39.8% to 79.1% in the tables of arXiv version 1, with one system given as 40.3% in the text and 50.3% in the table; judge agreement with humans on factual support r = 0.62. Five authors at Salesforce AI Research, one at Microsoft Research. arxiv.org/abs/2509.04499 (Accessed: 17 October 2026)
  15. [Tier A] Allaham, M. and Diakopoulos, N. (2026). “Synthetic Sources?: Auditing Generative Search Engine Citations for Evidence of AI-Generated Sources.” Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 9(1), pp. 142 to 154. DOI 10.1609/aies.v9i1.43129. 712 queries, four engines, about 16% of cited sources AI-generated. ojs.aaai.org/index.php/AIES/article/view/43129 (Accessed: 17 October 2026)
  16. [Tier A] Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W., Koh, P. W., Iyyer, M., Zettlemoyer, L. and Hajishirzi, H. (2023). “FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation.” EMNLP 2023. DOI 10.18653/v1/2023.emnlp-main.741. Atomic facts as the unit of checking. aclanthology.org/2023.emnlp-main.741 (Accessed: 17 October 2026)
  17. [Tier A] Dong, X. L., Berti-Equille, L. and Srivastava, D. (2009). “Integrating Conflicting Data: The Role of Source Dependence.” Proceedings of the VLDB Endowment 2(1), pp. 550 to 561. Detecting copying between sources. doi.org/10.14778/1687627.1687690 (Accessed: 17 October 2026)
  18. [Tier A] Hannák, A., Sapiezynski, P., Molavi Kakhki, A., Krishnamurthy, B., Lazer, D., Mislove, A. and Wilson, C. (2013). “Measuring Personalization of Web Search.” WWW 2013, pp. 527 to 538. Parallel control accounts to separate noise from personalisation. doi.org/10.1145/2488388.2488435 (Accessed: 17 October 2026)
  19. [Tier A] Cameron, A. C., Gelbach, J. B. and Miller, D. L. (2008). “Bootstrap-Based Improvements for Inference with Clustered Errors.” Review of Economics and Statistics 90(3), pp. 414 to 427. Inference with few clusters. doi.org/10.1162/rest.90.3.414 (Accessed: 17 October 2026)
  20. [Tier A] Kakimov, R., Gillham, J., Bejtic, N. and Tan, X. (2026). “Auditing Citation Behavior in AI-Generated Search Summaries: A Framework and a Case Study of Google AI Overviews.” Proceedings of the 39th Canadian Conference on Artificial Intelligence (Canadian AI 2026), PMLR 318, pp. 224 to 235; double-blind review according to the conference call for papers. 2,597 queries on health, finance, legal and safety topics from MS MARCO; up to 100 organic Google results collected per query as the comparison list; 52.72% of 39,673 citation events outside that list; country, language and dates of collection not stated. Two of the four authors work for Originality.ai, whose detector of AI-written text the study uses; the paper has no conflict of interest statement; the author order on the PMLR page differs from the PDF. proceedings.mlr.press/v318/kakimov26a (Accessed: 23 October 2026)
  21. [Tier A*] Schulte, J., Bleeker, M. and Kaufmann, P. (2026). “Don’t Measure Once: Measuring Visibility in AI Search (GEO).” Preprint, University of St. Gallen. The first author is also affiliated with Aurora Intelligence. arxiv.org/abs/2604.07585 (Accessed: 12 October 2026)
  22. [Tier A*] Zhang, K., He, X. and Yao, J. (2026). “From Citation Selection to Citation Absorption: A Measurement Framework for Generative Engine Optimization Across AI Search Platforms.” Preprint. arxiv.org/abs/2604.25707 (Accessed: 12 October 2026)
  23. [Tier A*] Fang, H., Tao, S., Chen, N., Chang, K.-X. and Sakai, T. (2025). “Do Large Language Models Favor Recent Content? A Study on Recency Bias in LLM-Based Reranking.” Preprint. arxiv.org/abs/2509.11353 (Accessed: 12 October 2026)
  24. [Tier C] Martinez, O. (2026). “Optimizing Visibility in Generative Engines: A Critical Survey of Generative Engine Optimization (2023-2026).” Preprint, 15 July 2026; independent critical review of 45 studies; seven-part visibility vector; table of metrics with the column “What it does not establish”; measurement protocol (estimand first, seven to eight repetitions, three to five paraphrases, untreated baseline and placebo, human validation); drift as an object of study. arxiv.org/abs/2607.14035 (Accessed: 17 October 2026; title checked on 22 October 2026)
  25. [Tier A] Strauss, I., Yang, J., O’Reilly, T., Rosenblat, S. and Moure, I. (2026). “The attribution crisis in LLM search results: Estimating ecosystem exploitation.” Data & Policy 8, e15 (Cambridge University Press). DOI 10.1017/dap.2026.10064. About 14,000 logged conversations, eleven models, March to April 2025; 15.6% of answers skipped web search; Gemini 34% and GPT-4o 24% without retrieval; 92% of Gemini answers without clickable citation. Preprint: arxiv.org/abs/2508.00838. doi.org/10.1017/dap.2026.10064 (Accessed: 17 October 2026)
  26. [Tier A*] Miller, E. (2024). “Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations.” Preprint, 1 November 2024. Clustered standard errors up to about three times the naive ones. The author works for Anthropic. arxiv.org/abs/2411.00640 (Accessed: 17 October 2026)
  27. [Tier A*] Zhao, H., Hao, M., Li, Z., Li, Y., Liu, Z., Qin, B., Cheng, Y., Fan, C., Zou, H., Chen, J. and Li, S. (2026). “Benchmarking and Training Reward Models for Cross-Source Evidence Grounding in Search Agents.” Preprint, 13 October 2026. Checks whether cited support traces to at least two independent evidence roots; applied to 256 production trajectories of the Tencent Yuanbao search assistant. Authors at Tencent, Peking University and Renmin University of China; Yuanbao is a Tencent product. arxiv.org/abs/2610.17827 (Accessed: 18 October 2026)
  28. [Tier A*] Chen, M. and Koudas, N. (2026). “What Drives AI Citations? Content, Retrieval and Model Knowledge in Generative Engine Optimization (GEO).” Preprint, University of Toronto, 10 October 2026. Answers collected through the engines’ APIs, for ChatGPT with gpt-5.4-nano and web search, 24 June to 21 September 2026; replay through the same API, 13 runs on 11 days up to 7 October 2026: 100.0% of 22,315 cited URLs among the pages its searches returned. In the study, 46% of ChatGPT’s mentions of brands not named in the question were supported by no cited page naming the brand. English questions about the US market, written for organisations that use the authors’ platform (in the main set, a US fibre internet provider); the authors note that the API need not behave like the consumer application. The answers were collected by an academic monitoring platform operated by the authors; the authors thank ktau.ai for their assistance with the data in their analysis. arxiv.org/abs/2610.15033 (Accessed: 21 October 2026)
  29. [Tier D] EBU and BBC (2025). “News Integrity in AI Assistants: An international PSM study.” European Broadcasting Union and BBC, October 2025; press release 22 October 2025 (ebu.ch/news/2025/10), which describes sourcing problems as missing, misleading or incorrect attributions. Coordinated by the EBU and led by the BBC; 22 public service media organisations in 18 countries and 14 languages; free versions of ChatGPT, Copilot, Gemini and Perplexity; answers collected in late May and early June 2025 and judged by journalists against five criteria; 45% of answers with at least one significant issue, 31% with serious sourcing problems. A research report by the broadcasters, not peer reviewed; funding not stated. ebu.ch/research/open/report/news-integrity-in-ai-assistants (Accessed: 23 October 2026)
  30. [Tier E] OpenAI (2026). “GPT-6 and Intelligent UI for everyone.” 7 October 2026. Rollout to Plus, Pro, Business and Enterprise on 7 October, to Free and Go from the following day; logged-out use is not mentioned. Vendor announcement about its own product. openai.com/index/gpt-6-for-everyone (Accessed: 16 October 2026)
  31. [Tier E] Loktionova, M. (2026). “Why 62% of AI citations don’t lead to brand mentions [Study].” Semrush, with Kevin Indig and Growth Memo, 9 June 2026. 3,981 domain appearances, 115 prompts, 14 countries, four engines. Published by a vendor whose tool collected the data. semrush.com/blog/the-ghost-citations-study (Accessed: 12 October 2026)
  32. [Tier E] Southern, M. G. (2026). “ChatGPT Traffic Drop Appears To Be A Tracking Problem.” Search Engine Journal, 15 October 2026. Trade press; reports figures and statements from Graphite, LLM Pulse, Adobe (Cedric Huesler), Peec AI and Profound, all of which sell or use AI search analytics; OpenAI’s confirmation of a bug is reported second-hand. searchenginejournal.com/chatgpt-traffic-drop-appears-to-be-a-tracking-problem (Accessed: 23 October 2026)
  33. [Tier E] Schwartz, B. (2026). “OpenAI Drops UTM Parameters From ChatGPT Results (GPT-6).” Search Engine Roundtable, 14 October 2026, updated 15 October 2026. Trade press; quotes Cedric Huesler’s posts “There’s no tag and no referrer” (14 October) and “Relieved. It was a bug.” (15 October); the confirmation by OpenAI, the 95% to 100% figure and the referrer on 15 October are not in this report seroundtable.com/utm-parameters-missing-chatgpt-42299.html (Accessed: 23 October 2026)
  34. [Tier E] Blyskal, J. (2026). “ChatGPT’s missing UTM tags: what our research shows.” Profound, 14 October 2026. Vendor of AI visibility analytics; no sample size stated; tagged cited answers about 98.5% (8 October, 18:00 UTC), about 75% (9 October), about 67% (14 October before 19:00 UTC); Intelligent UI share about 25% (9 October) and about 34% (14 October); tag loss began about 18:30 UTC on 8 October; the page was later extended without an update notice: tag back in about 99% of cited answers at 06:30 UTC on 16 October. tryprofound.com/blog/chatgpt-missing-utm-tags (Accessed: 23 October 2026)
  35. [Tier E] Haigler, N. (2026). “The Listicle Window Is Closing in AI Search: 30% Decline MoM.” Seer Interactive, 25 February 2026. Agency; more than two million citations, November 2025 to February 2026; ChatGPT citations of listicles down 30% from December to January; share 17.8% in November to 15.5% in January. seerinteractive.com/insights/the-listicle-window-is-closing-in-ai-search-30-decline-mom (Accessed: 17 October 2026)
  36. [Tier E] Konitzny, D. (2026). “GPT-5.6: product pages vs listicles.” Peec AI research notes, August 2026. Vendor of AI visibility analytics; no sample size or exact window stated; share of listicles among retrieved pages 15.47% to 8.19% after GPT-5.6. docs.peec.ai/research/gpt-56-product-pages-vs-listicles (Accessed: 17 October 2026)
  37. [Tier E] Resoneo (2026). “What ChatGPT pulls, what it shows, what it cites.” 1,249 answers with search (682 instant, 567 thinking), captured in July 2026 on real accounts and in logged-out sessions; cited URLs compared with the search results in the answer stream; the field naming the search engine behind each result last seen on 21 July 2026. Published by a search marketing consultancy that sells search consulting. think.resoneo.com/chatgpt-retrieval (Accessed: 21 October 2026)
  38. [Tier E] IAB (2026). “Measuring Visibility in the AI Era.” Interactive Advertising Bureau, August 2026, 36 numbered pages. Guidelines from the IAB AI Visibility Measurement Framework Working Group, whose members include brands, agencies, publishers and providers of AI visibility measurement; part of IAB’s Project Eidos. Defines visibility metrics, a directional and a decision-grade tier, required provider disclosures at two levels, and practices for baselines and resets; downstream attribution is left to a later framework; no data of its own. Published by an industry body. iab.com/guidelines/measuring-visibility-in-the-ai-era (Accessed: 22 October 2026)
  39. [Tier E] Google (2025). “AI Overviews and AI Mode in Search.” Google, May 2025, p. 3. The operator’s own description of its product. search.google/pdf/google-about-AI-overviews-AI-Mode.pdf (Accessed: 23 October 2026)
  40. [Tier E] Microsoft Bing (2026). “Introducing AI Performance in Bing Webmaster Tools Public Preview.” Bing Webmaster Blog, 10 February 2026. A report for site owners on total citations, average cited pages, grounding queries (a sample of citation activity) and page-level citation activity. The operator’s own description of its product. blogs.bing.com/webmaster/February-2026/Introducing-AI-Performance-in-Bing-Webmaster-Tools-Public-Preview (Accessed: 23 October 2026)
  41. [Tier E] OpenAI (undated). “Web search.” OpenAI API documentation, no date shown, read on 23 October 2026. The sources field against inline citations. The operator’s own description of its product. developers.openai.com/api/docs/guides/tools-web-search (Accessed: 23 October 2026)
  42. [Tier E] Long, C. (2026). “OpenAI Has Tripled Their Crawl of the Web: An Analysis of 7B+ Log Files.” Botify blog, 23 April 2026. Guest article by the co-founder of an SEO agency, written with the Botify team, on the blog of a vendor of log analysis; roughly 7 billion log files, as the post calls them, from Botify’s enterprise clients, November 2024 to March 2026; number of sites and verification of the bots not stated. botify.com/blog/openai-tripled-web-crawl (Accessed: 24 October 2026)

Evidence ledger

Where the key figures appear in the cited works.

Figure in this articleSourceLocation and basis
GPT-6 for paid tiers on 7 October, Free and Go from 8 OctoberOpenAI, 2026Announcement, opening paragraphs; logged-out use not mentioned
55,393 queries; 7,583 overviews, 7,491 evaluated; 98,020 claims; 11.0% not backed (6.98%, 2.66%, 1.39%); 41.9% fully backed; activation 13.7% and 64.7%; claim extraction recall 83.23% in a check of 100 overviewsXu, Iqbal and Montgomery, 2026Abstract and results sections; data 13 March to 21 April 2026; limits of extraction and retrieval in the methods section; extraction check in Section 3.3.3
Citation accuracy about 40% to 80% (summary); 39.8% to 79.1% in the tables, 31.4% for an extra web search setting; 40.3% in the text against 50.3% in the table for one system; source necessity 50% to 69% for generative search engines; judge agreement r = 0.62Venkit et al., 2026Abstract; Table 1; human validation section (100 verification tasks); results as of 27 August 2025; metric definitions in the evaluation section (citation accuracy and thoroughness share a numerator and differ in the denominator)
Seven-part visibility vector; “What it does not establish”; seven to eight repetitions, three to five paraphrases; drift as object of studyMartinez, 2026Section 3.2; Table 3; Sections 11.1 to 11.4; Section 13.5. The repetition figure is derived from Schulte, Bleeker and Kaufmann, so the two are one origin on that point
15.6% skipped search; Gemini 34%, GPT-4o 24% without retrieval; 92% without clickable citation; about three uncited relevant pagesStrauss et al., 2026Abstract and results; 13,929 observations from LMArena logs, March to April 2025
About 16% AI-generated cited sources; 712 queriesAllaham and Diakopoulos, 2026Abstract
Yes or no check for at least two independent evidence roots; 256 production search trajectories of Tencent YuanbaoZhao et al., 2026Introduction; Table 2 (criterion L3); Section 5.1. The paper does not state the period of the logs
Clustered errors up to about three times the naive onesMiller, 2024Section on clustered standard errors; ratio 3.05 on the DROP example
Tag loss window 8 to 15 October; bug confirmed to Huesler on 14 October; fix from 15 October, 95% to 100% recovered; direct URLs in 42% against 8% to 10% of one million prompts; referrer still missingSouthern, 2026; Schwartz, 2026Search Engine Journal, 15 October 2026, statements attributed to Cedric Huesler (Adobe) and Tomek Rudzki (Peec AI); Search Engine Roundtable update of 15 October 2026 (Huesler’s posts “There’s no tag and no referrer” and “Relieved. It was a bug.” only; the confirmation, the 95% to 100% figure and the referrer on 15 October are from Search Engine Journal). Both relay statements by Huesler on LinkedIn: one origin, not two confirmations
Tagged cited answers about 98.5%, 75%, 67%; Intelligent UI about 25% and 34%; tag loss from about 18:30 UTC on 8 October; panel links still tagged; tag back in about 99% at 06:30 UTC on 16 OctoberBlyskal, 2026Profound blog, dated 14 October 2026 and later extended to 16 October without an update notice (read 23 October 2026), section headings by date and time; no sample size
Listicle citations down 30% December to January; share 17.8% (November) to 15.5% (January)Haigler, 2026Seer Interactive, 25 February 2026; ChatGPT only; URLs containing “best” or “top”
Listicles 15.47% to 8.19% of retrieved pages after GPT-5.6Konitzny, 2026Peec AI research note, August 2026; no sample size
252,000 trials, six models, 18 content factors; relevance and list position strongest for being cited firstVishwakarma, Kumar and Jamidar, 2026Abstract; design section (two sources placed in the context, brands anonymised, order counterbalanced)
Visibility as a distribution rather than a single figureSchulte, Bleeker and Kaufmann, 2026Abstract
61.7%; 3,981 domain appearancesLoktionova, 2026Section “1. 62% of citations don’t lead to brand mentions in AI answers” and section “Methodology”; base: all domain appearances (13.2% cited and mentioned, 25.1% mentioned only; about 82% among cited appearances by our calculation)
ChatGPT: cited 87%, mentioned 20.7%; Gemini: cited 21.4%, mentioned 83.7%Loktionova, 2026Section “2. Every AI engine behaves differently”; Gemini based on appearances (link, name or both); for ChatGPT only the 20.7% stated per answer, no basis given for the 87%: indicative only
51.5% and 74.5%Liu, Zhang and Liang, 2023Abstract and Section 4.2; averages across Bing Chat, NeevaAI, perplexity.ai and YouChat
56% and 72%Thakur et al., 2025Abstract and Section 4.2 (confusion matrices)
57%Wallat et al., 2025Abstract (“up to 57%”), Sections 5.1 and 5.3, results figure (Figure 6 in the authors’ copy of the ICTIR version, Figure 3 in the arXiv preprint), assumption in Section 6; Command-R+, 1,444 questions from NaturalQuestions; short statements from earlier answers appended to other documents; condition: the document replaces one the model had predicted as relevant but never cited; 273 of 476 cases in which the statement came back
Absorption measures are observationalZhang, He and Yao, 2026Section 5.2, Equation 2; described in Section 5.3 as an observational proxy
100.0% of 22,315 cited URLs among retrieved pages (replay); 46% of mentions of brands not named in the question without a cited page naming the brandChen and Koudas, 2026Section 3.1 (collection, 24 June to 21 September 2026); Section 6.1 (Replaying the Engines; 13 runs on 11 days up to 7 October 2026); Section 6.5 (The Model’s Prior); paragraph “Limitations” (API, English questions about the US market); gpt-5.4-nano through the API
1,249 answers; 85% and 31% of citations outside the search results; two explanations; field last seen 21 July 2026Resoneo, 2026Section “What gets opened gets cited”, subsection “Some URLs are cited from memory”; section “Method: how we know” (682 instant, 567 thinking answers; real accounts and logged-out sessions, July 2026); section on the change of 21 July 2026; page updated in August 2026
242 papers; quotationGreenberg, 2009Abstract (Results and Conclusion)
About 16.5% of a standard question setZhang and Choi, 2021Abstract; NQ-Open
“Every one of the seven AI ranking systems tested”Fang et al., 2025Section 1 (contributions) and Sections 4.1 to 4.5; seven listwise rerankers
Required provider disclosures at two levels; absence of a disclosure as a signal; citation as a link or named reference; share of voice as a share of all brand mentions in a category; citation rate read as being relied on; baseline reset with trigger and date, separate seriesIAB, 2026PDF of August 2026, pp. 14, 15, 17, 27 to 30, 33 to 34 (printed page numbers)
22 organisations, 18 countries, 14 languages; more than 3,000 answers; 31% with serious sourcing problems; missing, misleading or incorrect attributionsEBU and BBC, 2025EBU press release of 22 October 2025; report pp. 3, 7 to 10 (sourcing problems include information not supported by the cited source, p. 10), read in a copy hosted by a third party and matched against the EBU page and press release
AI Overviews “include links to web content that supports the information presented in the overview”Google, 2025PDF of May 2025, p. 3
Average cited pages “does not indicate ranking, authority, or the role of any page within an individual answer”Microsoft Bing, 2026Bing Webmaster Blog, 10 February 2026, section on average cited pages
Sources “returns the complete list of URLs the model consulted”; “The number of sources is often greater than the number of citations”OpenAI, undatedAPI documentation, section on sources
2,597 queries; 52.7% of citations (20,917 of 39,673) to URLs not among the up to 100 organic Google results collectedKakimov et al., 2026PDF pp. 4 (Definition 1), 7 (top 100 organic results) and 8 (52.72% of citation events); proceedings pp. 227, 230 and 231
ChatGPT-User events down 28% (1 December 2025 to 14 March 2026 against the period before); OAI-SearchBot up 3.5-fold since GPT-5 (August 2025)Long, 2026Botify blog, 23 April 2026, sections on ChatGPT-User and OAI-SearchBot; about 7 billion log files, November 2024 to March 2026; explanations offered as possibilities

Other figures are stated in the abstract or title of the cited work.

Tier legend. [Tier A] peer-reviewed; [Tier A*] preprint, review pending; [Tier C] independent review of ten or more studies, not yet peer reviewed; [Tier D] study with a documented method by an industry or professional body, not published by a vendor; [Tier E] vendor, platform, agency, industry body or trade press source, reported as a description of its own product, as its own requirements or as a market observation, with its commercial interest or that of its quoted sources disclosed in the entry. These labels describe publication status and source type, not a ranking of evidential strength.

Method. This article is a synthesis of published research. Each published figure was checked against the original source on 12 October 2026, the OpenAI announcement on 16 October 2026, and the sources added after a comparison with current research on 17 October 2026; the definitions of citation recall and of DeepTRACE’s measures, and DeepTRACE’s results date, were checked again on 18 October 2026, Chen and Koudas (2026) against the full text and Resoneo (2026) against its page on 21 October 2026, the IAB framework against the full document on 22 October 2026, and Kakimov et al. (2026), the EBU and BBC study, the platform documents and the reports on the October tracking problem on 23 October 2026, and Long (2026) on 24 October 2026. Besides research, the search before publication covered industry bodies, platform documentation, vendor research, the trade press and the sources AI systems cited in our own runs for questions close to this article. Vendor and agency figures are treated as descriptions of the sample they report, not as validated estimates for production systems in general.

Literature search on independence. On 18 October 2026 we ran seven queries through a web search tool, restricted to arXiv, the ACL Anthology, the ACM Digital Library, OpenReview and the AAAI proceedings, combining generative search, AI search or answer engine with source independence, shared origin, syndication, provenance, redundancy, source dependence, copy detection and evidence root, and three targeted follow-ups. Of 70 hits, all titles were screened and 11 papers read in abstract or full text; included was work that traces pages cited by a deployed AI search system to a shared origin or grades their independence. One paper met that rule (Zhao et al., 2026), and one could not be read. This is not a systematic review; the queries and each screening decision are part of the audit record.

Audit record. Each claim in this article that rests on a source is recorded as a pair of claim and citation, with the judgement of support, the origin and the dates, in the audit record for this article. It was compiled during drafting and is not an independent check; a blind second check of a sample, and its result, are reported there.

Update log. Version 1.0, 24 October 2026: first publication.


About the Author

Manuel Hürlimann runs GaryOwl.com and the AI search consultancy octyl in Wil SG, Switzerland, and researches how AI systems select and cite sources. No commercial relationship with any vendor or agency cited here. GaryOwl has an interest of its own in the audit practice proposed in this article. About us · LinkedIn · manuel@octyl.io

Framework disclosure: Digital Authority Engineering (DAE) is GaryOwl’s own framework. This article uses the vocabulary of the cited research wherever it has one; the independence grading, the audit record and the two shares reported next to citation recall are GaryOwl proposals.

Navigate the DAE Framework: DAE Framework · DAE Glossary · The Five Gates of AI Citation (the same pipeline, organised by where a source is lost) · The Source Discipline Behind Every AI Citation · Root-Source Positioning

Scroll to Top