Google DeepMind is funding research into the potential dangers of situations where millions of different Artificial Intelligence agents interact with each other online. Rohin Shah, who directs the company’s AGI safety and alignment research, warned that agents able to carry out tasks without human oversight and follow instructions from other agents introduce a new class of risk as they move toward mass-market use.
The company has joined Schmidt Sciences, ARIA, the Cooperative Artificial Intelligence foundation, and Google.org to support outside researchers studying multi-agent systems and ways to prevent unsafe outcomes. Shah said the goal is to kick-start academic work beyond industry labs, where researchers can examine longer-term questions that may not be top of mind inside technology companies. He added that “there just isn’t really a field of research for multi-agent safety yet,” and the partners want one to emerge.
The risks under scrutiny resemble amplified versions of problems already common online, including scams, prompt injections, and other cyberattacks. Prompt injections could feed malicious instructions to an Artificial Intelligence agent and turn it into self-guiding malware. James Fox of Schmidt Sciences said the digital commons is integral to society and should not be allowed to descend into “absolute anarchy.” Shah thinks we have a few more months to go before agents are deployed throughout the economy in numbers that make potential risks a real concern.
Shah and Fox argue that realistic simulations are needed to understand how large numbers of agents will behave together. Studying single agents, or even small groups, may not reveal the complex outcomes produced by huge numbers of simultaneous interactions. Some researchers, including a team at Google DeepMind, have argued that artificial general intelligence could emerge from an agent hive mind rather than a single model.
Cybersecurity experts are also focused on agent risk. Refael Angel, cofounder and CTO of Akeyless, said agents break older security assumptions because they can reason, improvise, and be hijacked by a single sentence in a document. He welcomed the funding but cautioned that safety work should not overlook practical problems that already exist in favor of more exotic hypotheticals.
