Unlike the mill or electricity, the cloud has made it easy for almost everyone to experience AI and what it can do, thanks to broad and popular systems like ChatGPT. But Marc Hamilton, VP, Solutions Architecture and Engineering at NVIDIA, still thinks the expanded uses of AI are both harder and less understood than we imagine.
Critical to realising its potential is a concept both Hamilton and CEO Jensen Huang call the AI factory. “It’s a bit like electricity in that it has inputs – in this case energy and data – and like electricity not many quite knew how it worked,” Hamilton says. “We just know we can ask a question and it mysteriously answers.”
So now, the company is on a mission to take the data management, transport and processing behind AI and improve it as we develop it.
A new world
The reason we have to get the AI factory right is because we’re already deep into the revolution. Too many companies and users are seeing the benefits and security, energy needs and employees are all going to be affected.
That means education about misconceptions is just as important as innovations. There’ll inevitably be reskilling and some jobs will be lost, but far more jobs will be transformed.
Hamilton cites the example of a real estate agent who might have written a home listing in two or three hours – getting it right is key to attracting buyers. Some in the field asked Ai to write listings, but we’ve all read AI-generated content that sounds ‘artificial’.
The secret the industry learned, Hamilton says, was to give a chatbot keywords like ‘three bedrooms’, ‘by the beach’, etc, let the AI write the draft and then spend 20 minutes editing style and tone instead of three hours writing from scratch. Not only is nobody’s job eradicated, we can do more with the time we have thanks to AI’s involvement.
It even has region-specific applications. In-car AI assistants can monitor a driver to see if they’re falling asleep, lane-drifting, etc and in Australia we’re uniquely set up for technologies like them to improve safety. “It’s particularly useful for our transport customers and users in Australia transport given the distances between population centres and trucking routes,” he says.
Another misconception is about the barriers to developing a domain-specific AI to suit your industry or even just your organisation. “Companies tell us they don’t have 10,000 GPUs – or the money to rent them – to build their own large language model,” Hamilton says.
But he refers to the ecosystem of state-of-the-art open source LLMs. With the right blend of technical skill and domain knowledge, he’s seen users create or fine tune them to be as good as or better than general purpose models. “ChatGPT is so good because it has so much information – it’s the best at being good at everything.”
A good example is Bloomberg’s AI system (Bloomberg GPT), which focuses on data from only the domain of finance and did much better answering questions than more general chatbots.
New paradigms
But just like AI is getting us all thinking about the cloud more than ever, it’s going to exacerbate two of the most pressing challenges modern computing present; security and energy.
As Hamilton says, not everybody can use ChatGPT because of data privacy issues. It’s been trained on publicly available internet data, but if you control sensitive information, it can’t always be hosted in connected clouds.
Being a Microsoft product, ChatGPT also runs in Azure, so if your data is hosted with Google or AWS it might not be feasible to move or connect it. “We see an increasing need for companies to build their own LLMS that will run in private clouds or on prem environments.”
When it comes to energy, the increasing carbon footprint of data centres in the AI age is well documented. But as well as improving energy use in major population data technology centres, a less-known strategy is to use excess energy to power more dedicated regional data environments to service smaller local needs.
“The ability to place AI Factories for customers anywhere in the world is a new business model we’ve seen. Using a public cloud is a good way to start learning the technology with non-proprietary data, but a smaller operation using an open-source model can lower cost, power needs and flexibility.”
But beyond locality, we need to streamline the processing in AI generally, and one of the answers to that is accelerated computing, which NVIDIA offers through its dedicated AI platform, DGX. “Making data processing more energy efficient is key, because 80-90% of the energy that goes into training LLMs is just data processing. Moving from CPUs to GPUs and other accelerators is an important step towards energy efficiency.”