ABS cultivates new data sources to power its agriculture numbers

The Australian Bureau of Statistics is increasingly using third-party data – from government reporting to satellite imagery and farm software – in the production of agricultural statistics and insights for farmers.

Speaking to the iTnews Podcast, agriculture statistics program director Rob Walter said the bureau stopped using more traditional agricultural surveys in 2023.

Until then, it had run a combination of an agricultural census every five years, with smaller surveys in intervening years, to capture data that formed the basis of its statistics.

The statistics – one of the longest-running productions of the ABS – remain an important input for the preparation of Australia’s national accounts.

They also help to understand the impact of factors such as weather, land and water use, technology and improved farm management practices on livestock, broad acre crops and horticulture production.

Retiring the census

Walter said the switch from surveys to third-party datasets was driven in part by dwindling response rates and increased costs.

“We made the decision to cease large agricultural surveys and move to a range of other existing data sources, and that decision was basically because we’d been struggling with the ability to get good response rates from farmers and [with] the increasing costs that had entailed,” he said.

“That’s a challenge statistical organisations across the world have been facing, not just with agriculture but across a whole range of different types of surveys.”

One of the factors at play was the amount of time needed to complete an agricultural census or survey

The census went to 100,000 farmers, and the smaller surveys to 25,000 farmers.

Walter said completing an agricultural census took longer than the housing and population census that most Australians would be accustomed to.

“It probably took farmers longer to complete that agricultural census than it did for me to complete my population and housing census,” he said.

“That’s one of the challenges for farmers. It’s a pretty tough job, and the last thing they wanna do when they come inside at the end of the day is fill out a survey form, particularly some of that data is already collected in other reporting that they’ve done to government or other organisations.”

Regional demands

In addition, for ABS, the census and survey data increasingly was not detailed enough to meet emerging statistical requirements.

“The other reason we decided to make the change is because a lot of these other existing datasets that we were aware of are able to better produce regional statistics because of the additional detail [contained within them],” Walter said.

There is increased demand for data that shows what is happening at a regional level in the agriculture sector.

For example, it’s not just about understanding at a national level the total value and volume of wheat produced in Australia, but where it’s being produced and what impact climate change is having on region-by-region volumes.

Geospatial data – sourced from the likes of Digital Agricultural Services or DAS – combined with levy data collected from farmers by other parts of government – is particularly useful in this context.

“It gives us quite detailed data about broad acre crops in small areas, letting us produce very detailed regional statistics,” Walter said.

This overcomes a challenge with more traditional census data, allowing a more granular understanding of exactly where crops are grown.

“Agricultural properties are quite complex,” Walter said.

“They’re huge, and they’re often not just in one location. You might have some farm paddocks in one area, and then a bunch of farm paddocks 30 kilometres away as well.

“That’s why the satellite data from DAS is really valuable because it helps us understand with greater spatial detail where those crops are.

“In the past, we’d base statistics off a business address, which is often in a regional town, but the farm paddocks will be in that surrounding area.

“If you’re just using address to produce statistics, those statistics [about crops] will be associated to that regional town, which is obviously not right, but we didn’t have any other data to help us understand exactly where [the crops] are [grown].”

A collaborative approach to change

To make the change effectively, the ABS convened statistical working groups with various parts of the agricultural sector – starting with livestock and red meat, before branching out into horticulture and broad acre crops – to ensure statistical outputs were useful.

This also helped to ensure the most suitable third-party data sources were selected, and that the way they were aggregated produced accurate outputs.

“They were able to give us a lot of information about what the datasets were designed for, how they might be able to feed into our statistics, what the quality of them was, and what biases there might be in those data sets,” Walter said.

“With that knowledge, we could assess them, work with the custodians of those data sources to get access to them, and then develop methods that edited some of the weaknesses in those data sets, but also used the datasets together so that the strength of one dataset could offset a weakness in another.

“We did that in quite an incremental way so that we would develop something, share the results with the industry working group, they’d provide us feedback and we’d then go away and improve it and then bring it back [to them].

“We’re still working through that process. Some of the statistics that we published last year, we know we can make small improvements to those processes to make them more efficient or to improve the regional detail a little bit.

“Those ongoing partnerships and working groups are something we envisage will continue into the future to support our ongoing understanding of the data and how we’re using those datasets together.”

Emerging farm software data use cases

Software is increasingly being employed by farmers to help them make decisions.

The ABS sees value in this data, but particularly on an aggregate level, where it could be used to provide broader insights that help farmers uncover effective behaviours and actions that they can implement themselves.

“I think there’s a real opportunity to work with a wider range of farm software companies to bring together large volumes of data about a range of different factors, and provide that information back to farmers,” Walter said.

“Potentially, we could look at things like some of the sustainability practices that different farms are doing, connect that to economic data from the farms, and understand what practices are linked with greater resilience or greater farm profitability in drought conditions, for example, and provide that back.

“For example, we could approach wheat farmers in a certain region and say, ‘Farmers in your region that are farming the same sorts of crops as you that do this to have better profitability.’

“That gives all farmers information to improve how they might run their farm.”