Skip to main content
Deep Dive 30 Apr 2024 - 10 min read

Metrics meltdown: Avoid TV's 'colossal balls up' in podcast measurement warn agencies as fragmentation inhibits ad spend, buyers grapple with walled gardens, cross-platform attribution, 'surface level' Triton Digital ranker

By Kalila Welch - Senior Journalist

Australia is a global leader in podcast listenership, but the growth of local industry is being stifled by measurement that many suggest simply doesn’t stack up. Fully 41 per cent of media agencies basically overlooked podcast ads in 2023, with concerns around measurement, effectiveness, addressability, and standardisation being the key barrier to uptake. It all comes down to a lack of consensus, with publishers like Spotify and Acast building out their own systems rather than working towards a more collective approach. The Triton Digital Ranker is the closest there is to an industry-wide currency, but there’s contention over the suitability of listeners and downloads as key metrics, plus buyers say it lacks the context required to make informed investments. While there’s a diversity of opinions about the way forward, what buyers can agree on is that collaboration by all stakeholders will be key if progress in podcast measurement – and therefore share of budgets – is to be made. 

What you need to know: 

  • The Australian podcast market has the potential to be one of the strongest globally, but a lack of consensus and consistency in approaches to measurement is holding back growth.
  • According to the latest Infinite Dial Australia report, Australia nudged the US to become the top podcast per capita listening nation in 2023, yet the same year only 59 per cent of media agencies utlilised the channel as a “significant or regular” part of their active, per IAB Australia.
  • The absence of standardised measurement between publishers and platforms makes the jobs of media buyers difficult, with it near impossible to compare the performance of different titles – especially when they’re putting forward the data that best suits their own narrative.
  • While the monthly podcast ranker commissioned from Commercial Radio and Audio (CRA) from Triton Digital is seen as a good starting place, it offers only a surface level look at podcast performance and trends.
  • Spotify and Acast are both notable absentees from the ranker, opting to sell their podcast titles against their own insights and first party data.
  • The use (and definitions) of downloads and listeners is also a point of contention, though so far, there are few other options for tracking impressions or audience size.
  • Meanwhile, the likes of LiSTNR and Mamamia are supplementing the Triton Ranker with their own audience insights.
  • Media buyers point to standardisation as the solution, calling for the audio industry to collaborate on the development of a new and improved measurement currency. Without that, The Hallway’s Chris Murphy says agencies “can’t compare apples with apples”.
  • Atomic 212’s Lorraine Woods wants to see a “cost per completed listen” metric, along with CPA, attribution and standardised incremental reach.
  • Meanwhile, Spinach’s Ben Willee calls for the podcast industry to create something akin to the OMA’s MOVE if is to avoid the “colossal balls up” witnessed in the TV landscape with the entrance of global streamers.
  • Regardless of what the solution looks like, EssenceMediacom’s Stefan Boden says it all comes down to “consensus on industry standards”.

Now is the time where the audio industry should be saying 'let's get together to compete with global behemoths like Spotify and YouTube, so we’re really well positioned to have data and to have all of the pipes and infrastructure ready to go'. [Otherwise, just look at TV for an indication of a measurement] colossal balls up.

Ben Willee, GM, Spinach

The Australian podcast industry is by many measures a glistening success story – we’re the top per capita podcast consuming nation globally and our home-grown content dominates local podcast rankings.

Last year’s Infinite Dial Australia report from Edison Research saw Australia surpass the US in terms of podcast listenership to take the top spot. The research – commissioned by Australia’s peak commercial audio body, Commercial Radio and Audio (CRA) – found that 33 per cent of the population aged 12 and over listened to a podcast in the last week – two points higher than the US.

But the size of the opportunity has not yet fully translated into a burgeoning podcast ad market. And that’s due to one major bugbear – measurement.

As of the end of 2023, IAB Australia had the podcast ad market at $99.1 million – 37 per cent of the total $265.8 million spent on online audio advertising in Australia.

But the number of media agencies spending on podcast ads significantly or regularly dropped 11 percentage points to 59 per cent, reverting to 2021 levels according to the IAB Australia’s latest Audio Advertising State of the Nation report.

Media agencies ranked measurement and tracking as the top barrier preventing digital audio from being ‘a larger proportion of ad volume’ (45 per cent), followed by evidence of effectiveness (40 per cent), addressability (25 per cent), and standardisation of metrics (24 per cent).

As ever, industry is divided on the best way forward.

CRA and its members are wholly positive about the monthly metrics they commission from the Triton Digital’s Australian Podcast Ranker, while the likes of Spotify and Acast would rather keep things in their own walled gardens.

Neither approach is tracking for agencies, who in the absence of a comprehensive or comparable dataset find themselves running in circles to get a full picture of the opportunity and compare performance.

EssenceMediacom’s group investment director, Stefan Boden, describes the measurement issue as the “biggest challenge” faced by the CRA and its members, with the lack of development inhibiting advertisers and agencies from “fully [harnessing] the power of audio and podcasting”.

State of play: Triton ranker

The world of podcast measurement is, broadly speaking, a very fragmented place.

Monthly podcast rankers from Triton Digital capture a top-level view of the top 200 podcasts from ‘participating publishers’ in the Australian marketer, with deeper listener insights are locked away in the back end of audio platforms. Meanwhile, those with their own first-party data are writing their owns narratives.

Since its 2019 launch in the Australian market, Triton’s ranker has been the closest thing the podcast industry has to a comprehensive measurement currency. In total, 35 publishers participate in the ranker, including Southern Cross Austereo’s LiSTNR, ARN’s iHeart Podcast Network, Nova Entertainment, Nine, NewsCorp, NBCUniversal News Group, Shameless Media, Mamamia, SEN, The Chaser, Schwartz Media and the ABC.

The ranker is commissioned by CRA, and per the industry body’s outgoing CEO, Ford Ennals, it offers a “definitive view of what people are listening to”. He says it’s a “significant and helpful tool for the industry and indeed for media buyers”.

But critics say it provides only a superficial look at the state of podcast listenership in Australia, relying on metrics that don’t accurately reflect how buyers and advertisers invest in the channel.

Spark Foundry chief investment officer Lucie Jansen likens it to a music chart, offering only a “top-line view on what podcast titles could be a good fit for a particular brand”.

“Triton is less of a measurement tool, which would give more detail on impressions, reach etcetera, and is more of an indication of the audience scale versus other titles” she tells Mi3. “It is, therefore, more useful if you are choosing a title to sponsor.”

Likewise, general manager of The Hallway, Chris Murphy, positions the ranker as “a good barometer of what’s going on”, but says it shouldn’t be used in isolation.

Metric madness: what's in a listener? 

Triton’s monthly reports are derived from two key data points: monthly listeners and monthly downloads. According to Triton’s own definitions, the first figure refers to the number of single users that download a given title for immediate or delayed consumption, while the latter is the total number of unique file requests downloaded, including partial and progressive downloads. In accordance with the IAB’s Podcast Measurement Technical Guidelines version 2.1, both metrics are only counted when a minimum 60 seconds has been downloaded.

However, establishing a consistent measure (and understanding) of what constitutes a ‘listener’ is still proving a challenge. Mi3 received a variety of definitions from interviewees, with many believing this metric indicated a listener had listened through an episode.

By contrast, the IAB’s own definitions would suggest a listener refers to a unique user who downloaded at least 60 seconds of an episode.

So, if 100 people download an episode, does that equate to 100 listeners? Triton Digital SVP of measurement Daryl Battaglia says that’s “generally correct”. But the lack of consistency between platforms can make it complicated – “often there is no information to determine if a downloaded podcast episode was listened to,” he tells Mi3.

(Some would argue that media buyers have been less picky when it comes to broader digital metrics – viewability standards, for example, are pretty weak proxies to determine whether a person has seen an ad. Yet digital ad investment has soared over the last 15 years.)

Nevertheless, it’s the downloads figure that remains the more contentious of the two. While the rankings are based on the monthly listeners, the inclusion of the downloads figure can inflate the perceived engagement of a title – the common critique being that not all downloads eventuate in consumption.

“Once an audio file is downloaded onto a device, the ability to track engagement ends,” says Spotify’s JAPAC podcast account director, Sam Moles.

“Consequently, there is no certainty about whether the download translated into listenership or if the intended audience has taken further action from the ad.”

General manager and media director at Spinach, Ben Willee, agrees that downloads aren’t cutting it. They’re not “an effective and robust measurement for what someone has actually listened to”, he says. “It’s not the right metric because it doesn’t really tell us who is listening, and it doesn’t really give us insight into target audience reach.”

Until recently, the issue was exacerbated by the automatic downloads function on Apple Podcasts, which enabled automatic downloads for all shows that a user followed. Changes to the feature announced in Apple’s iOS 17 rollout in September have made automatic downloads more responsive to user’s actual listening habits. The corresponding decline in download figures globally has been welcomed by buyers.

Incomplete data-set

Regardless of whether listeners and downloads are the right metrics, there’s also the argument that the Triton Ranker is an incomplete dataset. Though it counts each of Australia’s key commercial radio players amongst its ranks, and a large share of Australian publishing business, there are still glaring gaps.

That’s likely based on two factors. First, participation in the ranker is pay to play, meaning the payoff might not be worth it for smaller independent publishers. Second, it’s arguably in the interest of publishers with first-party data to stick within their own walled garden.

Spotify is perhaps the most obvious publisher missing from the lineup. While the audio platform’s original content slate is narrowing, the tentpole titles that remain are frequently touted as some of the biggest titles globally, like The Joe Rogan Experience, Call Her Daddy, Anything Goes with Emma Chamberlain and Armchair Expert with Dax Shepherd. Locally, it’s also got comedy duo The Inspired Unemployed.

The platform attributes its absence from ranker to the strength of its “first-party data”, which it offers to agency and advertising partners through self-serve ad platform, ‘Spotify Ad Studio’.

“We believe the first-party data and depth of insights we provide podcasters in the form of listening audience, episode listening data and the opportunity to measure engagement and interaction through Polls and Q&A has significant value,” Spotify’s Moles, tells Mi3. “We don’t want this to get lost as part of an aggregation exercise and blending of data sets.”

After investing heavily to build out its walled garden, the platform appears to be veering further away from third party measurement, confirming last week that it had quietly let its IAB US and IAB Tech Lab membership expire. Locally, Spotify remains a member of IAB Australia. The audio company has said it will reevaluate its US membership later in the year. 

Then there’s Acast, which opts into the ranker by title. Its local lineup includes lineup The Guardian, Julia Gillard’s ‘A Podcast of One's Own’, Osher Günsberg’s ‘Better Than Yesterday’, and Mark Bouris’ ‘The Mentor’ and ‘Straight Talk’. But as a hosting platform and distribution partner for independent podcasters and publishers, rather than a publisher itself, the decision ultimately comes down to the content creators.

“Acast has never made a decision whether or not to support the Ranker – we leave that decision to the content creators we work with and represent,” says Acast AUNZ managing director Henrik Isaksson. “If our creators decide they want to be included in the ranker, we have always been supportive in providing the data logs to various rankers around the world.”

Ultimately though, the platform tends to push advertisers towards its own audience-driven solutions as, per Isaksson, "measuring the size of a podcast audience alone does not gauge the efficacy of a campaign or the influence that niche audiences can wield in fulfilling a brand's campaign objectives".

To each their own: how podcast publishers measure up 

Needless to say, there’s a diversity of opinions about what best practice podcast measurement looks like. Of course, publishers are most inclined to go with methodology that suits their own narrative, and with that comes a fair amount of finger pointing – the ranker doesn’t go deep enough and walled gardens evade standardised scrutiny.

Yet the sentiment coming from the publishing side of the industry is outwardly positive.  On the whole, publishers are confident that they’re giving advertisers and buyers what they need to invest in podcast effectively now and into the future. Problem is, that is at odds with the IAB's data that shows an 11 percentage point drop in media agencies investing.

LiSTNR: Triton lights the way 

In alignment with their industry body, the podcasting arms of commercial radio networks are a big proponent for the use of the Triton Digital ranker, in conjunction with the first party data derived from their apps.

Southern Cross Austero’s (SCA’s) LiSTNR sits firmly in that camp.

“Triton has become almost the common source that we all use,” says executive head of LiSTNR podcasts, Grant Tothill. “When we’re talking about monthly audience, we’re all measured the same way.”

“In Hamish & Andy's case, it's circa 900,000 to a million listeners per month. Then if you want it to work out what are the ad impressions served, you can click downloads.”

Audience demographics can then be segmented using the network’s ad tech and the first party data that goes with it.

“Then the other stuff that we start to look at with our own ad tech and everything else is trying to go; well, what does that audience profile look like?” continues Tothill. “All the different genres have different audience makeups and that's the next part of what we try to understand… because then it helps us to (a) create content for them (b) go and sell that audience.”

But while audience insights give publishers their own “advance” in market, he’s firm on the need for a source of truth like the Triton ranker to avoid “confusion” and “even the playing field”.

He says that while Spotify and Acast are for now sticking to “a data set that suits the story that they want to tell”, the industry would welcome them into the ranker because it would mean “everyone is standardised” – and “that helps everyone”.

Mamamia: Google gripe 

Outside of the big networks, there’s a strong cohort of indie publishers that are a part of the Triton ranker – and Mamamia is one of them. Like LiSTNR, the publisher leverages the Triton data in conjunction with audience insights derived from its own app and other platforms.

The media company’s head of product, Luca Lavigne, is optimistic about where the industry is headed. He says that podcast measurement “has come a really long way” in the last five to ten years and things are “still moving in the right direction”.

And, on the issue of standardisation, he's pretty relaxed – he reckons that market forces will eventually drive things in the right direction.

“In an ideal world, yes, there would be a centralised place that you can see every podcast, but I don't think we need to force our way there,” says Lavigne. “I think the free market will do its thing in that most buyers of audio and podcasts want to see where the shows they’re buying might stack up against the rest of shows in the country.”

His only gripe with the current state of measurement lies within the decentralisation of consumption data between different podcast platforms.

The gold standard, he says, would be for this data to feed back into the core RSS feed so that everything is in “one place”.

For that to happen, he says Google would have to play ball. The tech giant last year announced its plans to migrate Google Podcasts users to YouTube Music when the former was sunset (effective in the US this month), promising to build the latter into a single destination for audio.

While that investment is likely to be a “huge boon for the industry”, he says there needs to be pressure on Google to make its analytics available to publishers. At present, YouTube’s approach is to duplicate content by importing podcast RSS feeds, meaning downloads happen natively within YouTube’s own system, and analytics don’t flow back to podcast hosts.

“On YouTube’s platform at the moment, whether the podcast is audio or video, they rehost, which blocks publishers from getting analytics,” says Lavigne. “And they are the only major podcast platform to do that.” 

Spotify: 1st party solutions take the cake 

Spotify’s Moles outlines the need for full-funnel measurement with a “full suite of both first and third-party solutions”.

He says Spotify’s on-platform approach to measurement is powered by its dynamic Streaming Ad Insertion tech, and native Spotify Ad Analytics, through which the audio platform can “deliver pixel-based ROAS [return on ad spend] and attribution analysis” that are “on par with visual mediums”.

Built from the audio giant’s 2022 acquisition of pixel-based attribution platform, Podsights, Spotify Ad Analytics was repackaged last year as a free measurement solution that allows advertisers to track the performance of their audio advertising campaigns on any audio platform.

Beyond Spotify’s own measurement offering, Moles says there’s a need for third-party measurement companies to drive “cross-format advertising currency globally”, with Spotify supportive of the currency the IAB has developed as a “baseline” (despite its lapse in membership). 

Acast: Dealer's pick 

While not strictly a publisher, Acast is also in the business of shifting podcast inventory. But without its own first-party data to draw on, the platform takes a more audience-centric approach.

The starting point always revolves around identifying the key performance metrics and goals that the brand or agency considers to align with their particular campaign or objectives. This informs our recommendations, preferred audiences, talent and buying route,” says Isaksson.

“Many brands we collaborate with prioritise podcasting as a performance channel. For them, overall reach makes little difference, instead their focus lies solely on conversions or bricks and mortar sales.

“On the other hand, the most common use for podcast advertising in Australia is reach and engagement. In such cases we would recommend that the buyer should consider the type of content they are aligning with, reach within that specific audience vertical, show or type of ad unit that meets their objectives.” 

The focus should be on connecting with the right audience, no matter the size of the podcast. Scale is only one metric that should be taken into consideration when evaluating the audience, reach and efficacy of ad investments.”  

Buyer views: Fragmentation woes

While podcasting is powering, fragmented measurement and attribution practices are holding back investment in the eyes of media buyers.

At the core of the issues in the lack of consistency across the audio industry – per those interviewed for this piece, everyone is doing it their own way. 

“There's a lack of standardised metrics across different podcast platforms and hosting services,” says EssenceMediacom's Boden. That makes comparing performance consistently a challenge, "especially when reporting is not unified".

“Unlike other digital media, podcast platforms don’t provide detailed listener data which means it can be difficult to assess the effectiveness of activity – it’s not consistent to view cross-platform tracking,”  Boden explains. “Likewise, listeners access podcasts across various devices and platforms, tracking their behaviour and engagement accurately becomes complex. Attribution across platforms can be a challenge.”

As Spinach's Willlee puts it, buyers are “flying blind”.

Agencies "can't compare apples with apples"

Media buyers suggest they don’t solely look at CPMs, but cost per thousand downloads are still the basis of how most podcasts are traded, particularly in the case of programmatic – and building out the surrounding contextual factors is a harder task than it should be.

"When evaluating or reviewing podcast we also look at other metrics (outside of impressions and CPM) such as total listeners, audience profile, downloads, streams, contextual, audience or brand alignment and placement (pre, mid, post)”, Atomic 212’s national head of trading, Lorraine Woods tells Mi3.

With most of these factors locked away in walled gardens there’s little visibility across publishers and platforms, so per The Hallway’s Murphy, agencies “can’t compare apples with apples”.

Where to next? 

While agencies all tend to have their own ideas of best practice management, they seem to agree that there's a need for some kind of holistic, third-party measurement system that can inform both programmatic and direct (integrated) investment.

Woods would like to see the development of a currency that factors in "cost per completed listen" of podcast ads – much like a cost per completed video view. This, she says, would "ensure we only pay for audiences who have thoroughly listened to our ads within the podcast". Woods adds that the inclusion of cost per acquisition (CPA) and attribution measurement, "along with standardised incremental reach measurement" would also be ideal.

With a more simple take, Murphy envisions a "trifecta" of measurement that incorporates audience insights into existing data on monthly downloads and listeners, to create a "ubiquitous score". 

“A ubiquitous score would be amazing because it give you a well-rounded view of the opportunity, and the metrics that you provide of that score are relatively universal, irrespective of the platform," he says. 

Meanwhile, Willee points to the work done by the Outdoor Media Association with MOVE (Measurement of Outdoor Visibility and Exposure) as the blueprint. “I think the audio/podcast industry needs to do something similar," says Willee.

“The industry has the possibility to work with the various trade organisations to come up with ways that we can better understand what that listenership looks like. Whether that be a demographic and psychographic combination, it's really important for all of us to be able to do our jobs better,” he says.

For Willee, it’s down to audio publishers to come together if they want to avoid the “colossal balls up” witnessed in the local TV industry and “not get caught napping”.

“I think now is the time where the audio industry should be saying let's get together to compete with those global behemoths like Spotify and YouTube, so we’re really well positioned to have data and to have all of the pipes and infrastructure ready to go.”

Ultimately, media buyer consensus appears to suggest that growth in podcast investment hinges on development of a more unified stance on measurement by the podcast industry.

SCA's Tothill however, is not so sure. He thinks sellers can do a better job of educating buyers.

"We've seen podcasting grow, we're seeing more and more people start to spend their money on podcasts and getting great results. But I think we're only at the tip of the iceberg when it comes to educating agencies on what they need to understand," he suggests. 

But those that hold the purse strings want solutions that go beyond surface level – and work across the piste.

"There needs to be consensus on industry standards so that as a collective key stakeholders can come together collaboratively as podcast platforms, hosting services, advertisers, and industry bodies to develop standardised measurement – plus ensuring that it all complies with user privacy and regulations," says Boden. 

What do you think?

Search Mi3 Articles