The dramatic impact Google’s AI-written summaries are having on search results visibility for publishers has been revealed through exclusive new research.
An investigation led by The Press found that AI-written summaries returned nearly a quarter of news-related search queries in mid-May in the US, with the result that organic links to publisher articles were greatly pushed down the page.
The research suggests that if this decline in search relevance is matched by an equivalent decline in search rankings, it would have a “devastating” impact on publisher clicks for these key search terms.
Google insists that links included in AI Digests generate more clicks, but has declined to share any data on this. Research published today suggests that in many cases AI Summaries contain no links at all (see examples below).
[Read more: Google and publishers: An unpredictable animal that could eat you at any time]
Google AI summaries push publishers from the top search result to a full scrolling page down
The press and a consortium of leading publishers commissioned SEO agency Autoritas to analyze the impact of AI Insights, formerly called Search Generative Experience (SGE), on their search visibility. This was a joint investigation with Ricky Sutton’s Future Media newsletter.
Content from our partners
The research was conducted by conducting search queries in the United States in mid-May, when AI Briefings were being widely rolled out by Google in the country. Since then, Google has scaled back the scope of AI due to widespread concerns that have been raised about the accuracy of results.
But the research does offer some insight into the potential impact AI Summaries will have if/when Google rolls them out more widely again.
The research looked at 3,300 of the most relevant search terms for major publications and found that AI-written abstracts were offered as the top result in nearly a quarter of cases (23.7%).
From “Britney Spears” to “travel updates” to “latest news,” the research looked at the search terms that publishers said drive the most traffic to Google.
In some cases, Google AI Summaries were automatically displayed, and in others a portion of the summary was displayed and readers were invited to expand on it.
In cases where an AI-written answer was provided by Google, the number one search result – which previously would normally have been held by a publisher – dropped by an average of 980 pixels, or one page scroll.
In terms of relevance, AI Summary dropped the first search result to roughly the equivalent of position five, or lower, in the organic search results.
The big unknown is whether a publisher will get clicks if referenced in an AI Summary. Google has declined to disclose data on this, and AI Digests are evolving all the time, so Press Gazette is reluctant to make specific predictions about potential traffic drops. But Google’s top organic result typically attracts about 40% of clicks according to some searches, with clicks dropping to about 5% for the fifth position in search results.
So a publisher that ranks first for a particular article on Google can currently see an 85% drop in the number of clicks on that article if the result is replaced with an overview for all readers. This will be especially bitter for publishers given that the summary written by AI is based on a summary of their own journalism (see examples below).
David Buttle has previously written for Press Gazette about why AI Digests appear to be an exercise in obtaining publisher content with little value given in return.
Google searches account for about a third of the traffic on most news sites. So the widespread adoption of AI Roundups could have a dramatic impact on publishers’ overall traffic (and therefore ad revenue).
The chart below shows the average number of pixels by which search results for various news-related search terms linked to external sites fell in AI Digests, compared to traditional organic results.
The research found wide variance in the ratio of search results that AI summaries return. About 55% of health-related search queries returned an AI summary versus 9% of movie/TV show queries.
The research found that Google was returning AI-generated summaries for even very new searches, effectively establishing itself as a publisher in its own right.
This footage from earlier in the year reveals that a search for “top stories” returned a Google summary, rather than links to the stories themselves. In the AI Digests version, the actual story links to publishers have been moved “below the fold” (or halfway) of the page, a position where they are less likely to drive clicks.
Google has even set up “AI Digests” to summarize editorial reports on contentious and sensitive legal stories, such as the cases involving former BBC newsreader Huw Edwards.
A search for “list of BBC presenters” returns the Press Gazette as the fifth ranked result on Google in the UK. When this search term was used on Google in the US during our study, the query returned a result dominated by an overview of AI that is almost entirely inaccurate.
The AI Summary results included: George Alagiah (deceased), Huw Edwards (then suspended), Dermot Murnaghan (not a BBC news presenter since 2007) and Nick Clarke and Sir David Frost (both deceased ).
Accurate and up-to-date news articles now sit far down the results (requiring two or three scrolls from the reader).
A search for “missing submarine” revealed how an AI summary replaced an article written by the publisher as the top search result for information about a submarine that sank while investigating the wreckage of the Titanic last year. The picture on the right (below) shows what happens with the output expansion written by the AI.
Google AI Overview: What Does It All Mean for Publishers?
Autoritas chief executive Laurence O’Toole led research into the impact of AI Insights on Google search. He said: “With only 23.7% of keywords generating AI Summaries, this study signals a much lower penetration of AI generating search results than we have seen in previous SGE research studies.
“For example, a similar Autoritas study of brand and product-related terms in March 2024 found a 91.4% penetration, and an earlier study of popular e-commerce terms in January 2024 showed an 86.8% SGE penetration.
“We believe this may be a matter of timing, as the data was collected over several days, the week beginning May 13, 2024. This coincides with Google’s announcement that it had begun rolling out AI Summaries to all users on Google I /O on 14. Can.”
Authorities found that Google AI Summaries are currently returning about 10% of keywords, up from 4% a few weeks ago when AI Summaries was the subject of widespread criticism for returning inaccurate and sometimes dangerous results.
O’Toole said: “What’s evident from a large number of private and public SGE studies is that Google has the ability to serve up an AI answer to 100% of queries if it wants to; it’s currently choosing not to.”
“Google will always avoid using a large language model to generate answers for certain sensitive topics, but the integration of generative AI into all areas of commerce is inevitable.
“Google is reported to say that it will not break down impressions and click data for AI Summary links in Google Search Console, so it will be very difficult to assess in the short term at least if Google’s assertion that these Summary of AI will generate more website clicks than traditional web results.
“This study shows that when AI Digests appear on the search engine results page, the top organic ranking slips on the page by about one page scroll. Our previous click-through rate studies give us clues as to how devastating it can be be a significant drop in rankings for a website.”
What does Google have to say?
Google’s blog post responding to widespread concerns about Google’s inaccurate AI Digests said: “People use AI Digests as a jumping-off point to visit web content, and we see that clicks on websites are of higher quality – people are more likely to stay on that page because we’ve done a better job of finding relevant information and useful websites for them.
“We see that links included in AI Summaries get more clicks than if the page were displayed as a traditional web listing for that query. As we expand this experience, we will continue to focus on delivering valuable traffic to publishers and creators.”
Currently, in the UK, a subset of UK Google users see AI Summaries by default for certain queries. No date has been announced for the full release.
Publishers concerned about submitting AI Summaries
The Press shared our research with the UK’s main publishers’ trade bodies, the News Media Association (which represents national and regional newspaper publishers) and the PPA (which represents the magazine and specialist publishing industry).
NMA chief executive Owen Meredith said: “Google’s stated mission is to ‘organise the world’s information and make it universally accessible’ by sending visitors to websites. Bringing GAI to search and AI Summaries that synthesize directly and present the information to the user risks discouraging users from clicking on the original links, in turn threatening the business model of those who invest in journalism and quality news and information.All of this is apparently without consultation, transparency, or permission remuneration to the original curators and rights holders of that information.
“This potentially dangerous move ultimately risks undermining the diversity and quality of news and information available online. In that case, we all lose.”
PPA chief executive Sajeeda Merali said: “As the voice of the publishing industry, PPA is committed to advocating for fair practices that ensure the value of our members’ original content is recognized and protected. We call on Google work collaboratively with publishers to develop solutions that support quality journalism and maintain the integrity of the publishing ecosystem.
“It is also in the best interests of consumers so that the information they seek always comes from a trusted source. The implementation of the Digital Markets Bill will be a priority for the next government and we look forward to seeing this implemented .”
METHODOLOGY
Six publishers shared nearly 500 of their top traffic-generating search keywords to help Authoritas understand the impact of AI Digests on traffic. They were: Press Gazette, Reach, DMG Media, Hearst, Future and Australian media and marketing publisher Mi3.
Autoritas used SGE Rank Tracker to generate thousands of searches using specified keywords. It then recorded instances where AI-generated results were triggered.
Email pged@pressgazette.co.uk to point out mistakes, give story tips or submit a letter for publication on our blog “Letters Page”.