The ABC implements data warehouse amid governance restructure

The Australian Broadcasting Corporation (ABC) has consolidated its data into Snowflake’s data warehouse as part of a three-phase project “to operationalise [its] data governance”.

The public broadcaster adopted the cloud-based platform with the goal of delivering “an easily accessible, centralised view of ABC’s critical data assets”.

The shift was one component of a three-year data governance acceleration program, with the platform playing a key role in cementing data “accountability and clearly defined responsibilities” across the organisation.

The program was unveiled by the ABC’s manager of enterprise data and analytics, Alison Burgis, during Gartner’s Data & Analytics Summit in Sydney last month.

“With the volume of data growing exponentially across industries, alongside the increasing occurrence of data breaches and the associated risk of reputational damage, it’s more important than ever that data is managed effectively and in compliance with law,” she told the audience.

While the ABC had a data governance function “for some time”, according to Burgis, the organisation was still “encountering significant data governance challenges across data, people, processes and technology”.

“In many instances, our approach to data quality issues was reactive in nature,” she said.

“Data discoverability was a challenge, and time-to-insight was simply too long.”

From a technology standpoint, Burgis noted that the ABC’s data was housed in multiple “siloed” platforms with no centralised data management or automation tools for data quality and metadata management.

However, according to Burgis, the people-related challenges were the most onerous.

“There was a lack of clear accountability and heavy reliance upon single points of failure for their data knowledge,” Burgis said.

“Furthermore, there were varying levels of maturity and data governance; a lack of data governance resources and a lack of expertise for such activities.”

These pain points were at the root of the ABC’s data governance acceleration program, a “practical roadmap to deliver incremental, long-lasting change”.

Model of accountability

ABC’s data governance initiative was structured around three key waves: foundation, enablement, and embedment.

The foundation phase focused on establishing the core frameworks, standards and processes, as well as a model for accountability among the broadcaster’s people.

At the structural level, this includes the establishment of a data governance council, which reports directly to ABC’s leadership team.

Supporting the council is a hub-and-spoke data operating model, where central governance is complemented by distributed responsibility across the organisation.

Within the “spokes,” there are 18 data owners who are accountable for data-related decisions within their own individual data domains.

According to Burgis, they effectively oversee the lifecycle management of their data, including its collection and alignment with the broadcaster’s standards.

“During the project, critical data assets were defined by each data domain with ownership, classification, risk and value,” she added.

Sitting adjacent to the owners are 15 “stewards”, who are “responsible for monitoring, cleansing and documenting” the data.

In Burgis’ words, they are “data custodians” who “often reside in technology and have a high level of skills in systems and tools”.

With the foundation in place, the program’s second phase saw the ABC establish a SharePoint-based portal to house its data governance artefacts, a glossary and information about meetings – referred to as forums – between the data stakeholders.

The final phase, embedment, Burgis referred to as simply a process of cementing “governance into the data lifecycle and daily activities”.

By design

Commenting on the program’s outcomes after 18 months so far, Burgis said it “met all project targets”, including setting up clear lines of responsibility and creating a cultural shift across the organisation.

This, she said, is evidenced by the data owners and stewards now being required to present their relevant data issues to the governance council.

In addition, she said, 4000 employees have completed customised introductions to data governance.

With these foundational milestones in place, the ABC is now looking ahead to embedding what Burgis calls “data governance by design”, placing it “front of mind from the outset, not an afterthought”.

“We’re actively working to become more proactive in our daily quality management and shifting governance to the left to identify data quality issues at creation and ingestion,” she said.

The team is also exploring opportunities for automation, particularly in data quality monitoring and metadata management.