The University of Queensland’s Bunya supercomputer is being upgraded with new GPUs and thousands more CPU cores, with project completion due in August or early September.
Bunya in the Polaris data centre
University of Queensland
Jake Carroll, CTO of the university’s Research School of Computing (RSC), told iTnews the additional GPUs will make “Bunya Phase 2.0” more suitable for “accelerated computing, machine learning, artificial intelligence and interactive visual computing.”
Carroll said the university wants Bunya to support as many kinds of workloads as possible, because “we want to help as many researchers accelerate their research outcomes as possible”, across many disciplines.
Hence the machine is running code “from almost every research domain including … the humanities, economics, molecular dynamics, genomics, weather and climate modelling, artificial intelligence and astrophysics,” he said.
While designed to be a balanced system, Carroll said Bunya was built with “some specific research communities in mind”.
“For example, UQ’s bioinformatics researchers make use of Bunya’s uniformly very high memory footprint per node.”
Bunya Phase 2.0 will feature a fleet of Nvidia H100 and L40 GPUs, and AMD Genoa CPUs. The system runs 200Gbps per node networking using Nvidia HDR InfiniBand smart edge switches.
“Because we are firm believers in fostering the growth of the multi-vendor GPU ecosystem and providing diverse options for different research use cases, we also have more AMD Instinct MI210 GPUs coming online,” Carroll said.
“AMD’s Instinct GPUs are gaining momentum in the scientific research community as a viable alternative as the software ecosystem grows stronger.”
Storage is provided by IBM ESS3500 Storage Scale parallel file systems, with all-flash NVME to accelerate I/O.
The H100 GPUs will use HBM3 memory, while the Genoa CPU uses DDR5 memory, which Carroll said provides “lower latency and higher throughput inside these nodes when moving data between components.”
Carroll said this upgrade is part of the Research School of Computing’s plan to have an “evergreen” supercomputer – one designed for iterative change, rather than large one-off purchases, which he said “can lack agility and stymie access to the very latest technologies.
“We operate on a five-year planning horizon for each new tranche of technology we put in place,” he said.
“We can expect that in five years, we will consider the value of these components that are getting commissioned right now and look carefully at the market to determine where we place our investments next to get the best value and research outcomes for UQ.”
One key metric the university will consider at the five-year mark will be “computational outcomes per watt of power consumed”.
Carroll explained: “More than ever, we must consider exactly how much power every new component uses and the value the power that goes in provides back to the researcher and the University.
“The days of power consumption without tight measurement and consideration for the environmental impact are long gone (and rightly so), in our view.”