Search and rescue: New study shows top ranking SEO links tumble, publishers face double digit traffic declines in Google flip from search gatekeeper to AI publisher
Google's quiet global rollout of "AI Overviews" in its search listings this year – in almost every market but Australia – has suddenly popped, turning 25 years of analytics and SEO on its head and likely to render the worth of much organic search activity a fraction of what it is currently. A new study commissioned and released today by an international syndicate of UK and US publishers –including MI3 and Future Media in Australia – the extent of AI Overviews being served by Google in its search results across thousands of keywords and compares referral traffic between search today and how Google is reshaping it for content discovery and monetisation with AI. The picture is not pretty. UK publishers, Hearst, Daily Mail and Trinity Mirror along with Press Gazette and Australia's Mi3 and Ricky Sutton's Future Media jointly commissioned software platform Authoritas to conduct the study. It shows top ranking links in countries where AI Overviews is live are tumbling down – and off – the page across all categories studied, and replaced by AI-created amalgamations. Worse still, referral links in categories that attract the highest audience yields for publishers appear to be most degraded. Here's a deep dive into what Australian brands and publishers are facing – when Google deems it a more palatable political and regulatory environment to launch. This piece also leads Ricky Sutton's Future Media edition today.
What you need to know:
- Google has hitched its Gemini AI large language model to its search cash cow – and the impact for publishers in terms of search ranking, visibility, traffic and revenue is profound.
- Analysis of thousands of search terms across major publishers including Daily Mail Group, Trinity Mirror and Hearst found Google’s new AI Overviews approach pushes top ranking publisher links down the page. Or off it entirely.
- Australia remains one of the last countries where AI Overviews has not been deployed. But that may change come September.
- A deep dive into the impacts by Future Media, with software platform Authoritas, plus publishers Daily Mail Group, Trinity Mirror and Hearst, as well as Press Gazette in the UK and the US, and Mi3 in Australia, unpacks what's coming rapidly down the track.
It’s six months and a day since a whistleblower alerted me to the catastrophic risks coming with Google’s no-click search called SGE, since renamed AI Overviews.
You can track my posts since, but the warning was loud and clear.
“Publisher traffic will collapse, and ad revenue will crater, making it the greatest threat to the media industry since the dawn of the digital age.
“This is Google’s frightening vision of the future. It’s no more Mr Nice Google.”
Smart arse SEOs sniggered and knew better. CEOs looked the other way. I was subjected to Cassandra Complex yet again, but you can’t give in.
What’s past is past, and the future still needs to be saved, so let’s focus on what we know, and what to do about it.
Today, in Future Media’s biggest investigation to date, I’m revealing the results of a collaboration with global publishers to analyse the impact AI Overviews will have on premium web traffic.
It’s the most comprehensive study into the implications of Google shifting its 91 per cent monopoly on search, into a zero, or few-click, model.
It’s a cash grab, an advertising Hail Mary, ahead of September’s antitrust case where the US Government will call on a judge to break the company up.
It leaves publishers on a tightrope for traffic and revenue, as AI Overviews turns 25 years of analytics and SEO on its head and renders it most likely worthless.
To get to real numbers, I worked with Daily Mail, Trinity Mirror and Hearst, as well as Press Gazette in the UK and the US, and Mi3 in Australia.
-
We ran searches on news articles and products, across more than 3,000 keywords.
-
We investigated traffic across 19 genres to assess the impact on news, sport, entertainment, health, food, and to assess how much views will fall for each.
-
Searches were conducted across news articles, online retail, and magazines, to ensure the findings gave insights across a gamut of publisher earnings.
-
Searches were done across every day of the week, and at all hours.
Today, the first tranche of results can be revealed.
Google search has changed. Google’s 25-year strategy to organise the world’s information has pivoted to answering the world’s questions.
Google has a 91 per cent share of global search so AI Overviews will rewrite the way content is discovered and monetised, and profoundly impact the publishing industry.
The search below is for the common search term ‘BBC news UK breaking’.
On the left is the traditional Google search engine result page (known as a SERP in SEO shorthand) which displays bbc.com in all the four the top-ranking results.
In AI Overviews on the right, the same query has Google’s AI summarising 11 BBC headlines and demoting the top-ranking BBC link to the bottom of the page.
In many cases, we found that links to news stories disappeared from the first page of the SERP altogether and required the user to scroll down to see them.
We measured the impact and found the average vertical drop of the top-ranking result was 980 pixels (roughly an entire desktop page scroll) on AI Overviews SERPs.
The demotion varied across the 19 categories we looked at, but every single one saw top-ranking links falling far down the search page.
Links to food, health, locations, and shopping, which have the highest advertising CPMs, appeared to be degraded the most.
This appears to suggest an attempt by Google to keep traffic in the most valuable advertising verticals within google.com, to retain a greater share of ad revenue.
Google already banks a 90 per cent of global search ad revenue, worth $175 billion last year.
That monopoly is subject to an ad antitrust case listed to be heard in a US court on 9 September.
In Top Stories, the traditional search returned links to top stories from a selection of top publishers, but in AI Overviews on the right, that had changed again.
On AI SERPs, Google summarised the headlines and shifted publisher links to the foot of the page.
The outcome is that Google hijacks traffic by not sending readers off google.com.
In January, I predicted that Google would do this to increase ad volume and yield on its owned and operated (O&O) sites.
Two days after this investigation was completed, my prediction came true when Google confirmed it would start serving ads on AI Overviews pages.
The Google blog revealed:
“We’ve heard that people find the ads appearing above and below the AI-generated overview helpful.
“Soon, we’ll start testing Search and Shopping ads in AI Overviews for users in the US.
“There’s no action needed from advertisers: Ads from existing Search, Performance Max, and Standard Shopping campaigns have the opportunity to appear within the AI Overview.”
Next, focus shifted to specific stories.
Google has been systematically pushing the boundaries of copyright since January 2014 by publishing paragraph snippets in SERPs.
Across the 19 genres looked at, Google used a paragraph to answer the reader’s query no less than 88.8 per cent of the time.
If snippets angered publishers, Google takes a far greater leap with AI Overviews, publishing more of the story, and sneakily hiding it behind consumer choice.
In the result on the right below, Google’s approach to write a précis of the article and push the top-ranking publisher link from CNN down the page continues.
But in the latest AI Overviews SERP on the right, it also includes a show more button. If a reader clicks it, Google then publishes further details to tell users the story.
The investigation found that the show more link was displayed on 78.4 per cent of AI Overviews pages.
There were also wide variations in which genres it appeared, providing more insights into Google’s strategy for revenue, and to avoid regulatory sanction.
Roughly 30 per cent of searches for news stories saw users offered AI Overviews.
Update: Since bad publicity over some AI generated search results recommending glue on pizza, the number of news stories featuring AI Overviews has fallen to almost zero.
News attracts lower yielding advertising rates as they are often commoditised, meaning they have less benefit for Google ad yields.
They’re also the most troublesome as Google is being pressured by governments worldwide to pay publishers for this type of content.
What was evident though is that Google has the ability to serve an AI answer for 100 per cent of queries if it chooses, meaning it can manipulate the market to its whim.
Independent SEO software platform Authoritas was commissioned to conduct the study.
Principal Laurence O’Toole said: “We’ve seen 100 per cent in some categories, or for certain types of queries.
“They’ve scaled it back a bit for now, but they're just getting started.
“Obviously, there are some areas, like porn, and anything that’s against its ethical AI principles, that they might steer clear of, but they have the capability.”
Google has been claiming for months that it is only experimenting with AI results as a replacement for search, as it quietly rolled it out to 150 countries.
It then went all in with its Hail Mary announcement that it would roll out to all America during its I/O conference last month.
This study began while AI Overviews was still in trial mode, during which time we discovered that AI replaced traditional search results 23.7 per cent of the time.
It found significant disparities in which genres were prioritised for AI results.
Again, the content verticals with the highest prevalence of AI results were the genres with the lowest risk, and highest advertising CPMs: Horoscopes 62 per cent, health 55 per cent, and shopping 43 per cent.
High volume and low yield categories like local, national, and international news, movies, TV shows, sport, and relationships were far behind.
Australia remains one of the last countries where AI Overviews has not been deployed.
Since Google formally announced AI Overviews, it’s been constantly tinkering with results pages.
The research has shown buttons moving, and wording changing, revealing that considerable research is going on in Google’s labs, as would be expected.
The most major changes came when AI SERPs began delivering nonsense results, including advising people to put glue on pizza.
Since then, Google has sharply pulled back on using generative AI to respond to searches around factual topics.
It means the design of SERPs is in flux, and Google has made no official announcements to publishers or content creators how long it will be before it settles down.
The actions confirm my prediction that Google is pivoting from a search engine platform into a publisher, to keep traffic on O&O pages where it can deliver ads.
Platformer’s Casey Newton wrote:
“Google’s idea for the future of search is to deliver ever more answers within its walled garden, collapsing projects that would once have required a host of visits to individual web pages into a single answer delivered within Google itself. By the end of 2024, they will appear at the top of results for 1 billion users.
“Google’s move to answer more questions on the search engine results page is simply a continuation of a long-standing practice.
“A quarter-century into its existence, a company that once proudly served as an entry point to a web that it nourished with traffic and advertising revenue has begun to abstract that all away into an input for its large language models.”
Reporting by Press Gazette revealed data from SEO tool Sistrix showing almost half of leading news publishers are suffering double-digit falls in search visibility.
Google said the changes are designed to remove low-quality, AI-generated junk sites that are cluttering search results.
However, BBC News fell 37 per cent and London’s Evening Standard 32 per cent.
Meanwhile, Google’s monopoly on search and search advertising generated $80 billion over the same period, up 15 per cent.
One intriguing finding was video.
Despite Google owning YouTube, and video ads being worth 30x as much as display ads, video appeared in just 0.4 per cent of AI SERPs.
One of the demands of the US Department of Justice’s antitrust case in September is forcing Google’s parent Alphabet to spin-off YouTube.
YouTube earned $8 billion last quarter. I agree with many commentators that Alphabet will proactively sever YouTube to protect its hero search business.
This would pitch the world’s two largest search engines against each other and satisfy some of the regulators’ monopoly concerns.
It might also explain why Alphabet didn’t unleash an army of lawyers when AI arch-rival OpenAI was caught scraping YouTube to train Chat-GPT.
Watch this space, we’ll find out in a few months…
Another finding was the way Google has changed which publishers rank highly in AI Overviews.
This is a major change as Google influences which publishers and content creators have been handpicked to receive the lion’s share of web traffic.
This derails years of work, and billions of dollars in SEO spending, by companies globally to win the top spots in organic search.
These changes have been done without transparency and run the risk of Google self-preferencing results for its commercial benefit.
The UK’s the Competition and Markets Authority (CMA) said:
“Google’s strong position at each level of the intermediation value chain creates clear conflicts of interest and a range of concerns about self-preferencing.”
It’s joined calls to break Google up
“Google has the ability and incentive to prefer its own businesses in ad tech intermediation.
“Google’s strong positions in publisher ad serving, SSP and DSP, as well as its unique access to Google’s ad inventory, means that each of these businesses faces a conflict of interest.”
Google asserts that AI Overviews reveals a greater diversity of websites in its results, but as the CMA says, Google’s secretive algos make this impossible to disprove.
*This study was conducted on desktop browsers not mobile, so although we expect the experience and effects to be similar, they may not be.