Nvidia CEO sees shift in datacentres to ‘AI generation factories’

Nvidia’s datacentre business has reported growth of 409% over a single year. The company’s CEO, Jensen Huang, puts this down to what he sees as a shift in the way datacentres are being run.

During the earnings call for the quarter ended 28 January 2024, Huang spoke about the shift in datacentres from general-purpose computing to accelerated computing. According to a transcript of the earnings call posted on Seeking Alpha, Huang believes general-purpose computing is running out of steam.

“You can tell by the CSPs [cloud service providers] and many datacentres, including our own, extending the depreciation from four to six years,” he said. “There’s just no reason to update with more CPUs when you can’t fundamentally and dramatically enhance throughput like you used to.”

Huang said it was no longer possible to sustain improved throughput using just general-purpose computing. He claimed that accelerated computing enables datacentre operators to improve energy efficiency and cost of data processing by a factor of 20 to 1.

Huang said generative artificial intelligence (GenAI) was a new application in the datacentre. “It is enabling a new way of doing software. New types of software are being created. It is a new way of computing. You can’t do generative AI on traditional general-purpose computing. You have to accelerate it.

“For the very first time, a datacentre is not just about computing data and storing data and serving the employees of a company. We now have a new type of datacentre that is about AI generation – an AI generation factory.”

“We now have a new type of datacentre that is about AI generation – an AI generation factory”

This demand for accelerated computing is expanding across different industry sectors, according to Huang. “Our datacentre platform is powered by increasingly diverse drivers – demand for data processing, training and inference from large cloud service providers and GPU-specialised ones, as well as from enterprise software and consumer internet companies. Vertical industries – led by auto, financial services and healthcare – are now at a multibillion-dollar level.”

Among the growth opportunities for the company is Nvidia AI Enterprise, a software platform for accelerated computing using graphics processing units (GPUs). It costs $4,500 per GPU per year and is predicted to generate $1bn in earnings annually, according to Huang.

“My guess is that every enterprise in the world, every software enterprise company that are deploying software in all the clouds and private clouds and on-prem, will run on Nvidia AI Enterprise,” he said.

Huang claimed that enterprises lack sufficiently large engineering teams to be able to maintain and optimise their software stack to run across public clouds, private clouds and on-premise datacentres. “We are going to do the management, the optimisation, the patching, the tuning, the installed-base optimisation for all of their software stacks. And we containerise them into our stack. Nvidia AI Enterprise is like an operating system for artificial intelligence,” he added.

The fourth-quarter filing showed that Nvidia grew its business by 22% in a single quarter to $22bn, an increase of 265% from a year ago. For fiscal 2024, Nvidia reported revenue of $60.9bn, an increase of 126%.

Commenting on the quarterly results, Nvidia chief financial officer Colette Kress said the 409% increase in the performance of its datacentre business was due to higher shipments of the Nvidia Hopper GPU computing platform, which is used for training and inference of large language models, recommendation engines and GenAI applications. InfiniBand, its high-speed interconnect, also contributed to growth in the datacentre business.

“In the fourth quarter, large cloud providers represented more than half of our datacentre revenue, supporting both internal workloads and external customers,” she said in a statement. “Strong demand was driven by enterprise software and consumer internet applications, and multiple industry verticals including automotive, financial services and healthcare.”

Due to US export restrictions, sales of datacentre GPUs in China “declined significantly”, Kress added.