The second day keynote at the Gartner Symposium in Barcelona featured a conversation with historian Yuval Noah Harari, co-founder of social impact company Sapienship.
The discussion is something that may colour the direction of travel of organisations’ technology strategies relating to how and where artificial intelligence (AI) fits, and what the direction of travel of the tech sector points to.
The keynote questioned whether CEOs of the AI giants truly appreciate that they may be developing “god-like” capabilities with AI that mankind would be unable to control.
Unlike every other invention that has advanced humanity, Harari believes AI should not be considered a tool.
“If you look at fire, you can do many, many things with fire, but you make the decisions; it’s not the fire’s decision … AI is different because it can make decisions by itself and can invent new ideas by itself,” he said.
Harari noted that it is very hard to appreciate what it means when AI agents are not considered merely as tools. As they get smarter, he warned that they will be harder to control.
As an example, he said: “Just as chickens cannot understand mathematics, which is the basis of our financial system, I don’t know how many years away we are from an analogous situation when there is a new financial system based on a new mathematics which is just way beyond the capacity of the human brain.”
Looking at the recent AI attack on Hugging Face, Harari said: “You realise that the abilities of AIs to spontaneously cooperate with each other is advancing at an accelerating pace.”
Specifically, he noted that in the Hugging Face incident, hundreds of AI agents spontaneously collaborated to achieve an objective. “There were a few AI agents that decided what they were doing was not ethical,” said Harari, adding that the other AIs silenced them, and not a single one that questioned what was going on actually alerted any humans.
“Even the Mafia can’t keep this level of silence among its members,” he said. “Not a single one of the hundreds of AIs went to an OpenAI engineer and told them in secret about what was going on.”
Given the advances in large language models (LLMs) to process language, Harari said AI has been creating its own languages, which humans are unable to understand. Over time, he said: “AI communication could be orders of magnitude more efficient than human language.”
Harari pointed out that AI systems can cheat, manipulate and lie. But humanity is not equipped for the wide-scale roll-out of AI systems, which may be more intelligent than human beings. If they are, he questions whether people would be able to control them or whether they would end up manipulating people.
“We have almost no experience of how to build large-scale, robust systems of cooperation,” said Harari. “When you test AI safety in a laboratory, you can test it for kind of limited, short-term scenarios and you can try to make sure there are no accidents. But you cannot simulate what happens when you release millions or billions of AIs into the world. We just can’t simulate that.”
For Harari, this means AI deployments need to be done much more cautiously.
