A Deep Learning Alternative Can Help AI Agents Gameplay the Real World

A new machine learning approach that draws inspiration from the way the human brain seems to model and learn about the world has proven capable of mastering a number of simple video games with impressive efficiency.

The new system, called Axiom, offers an alternative to the artificial neural networks that are dominant in modern AI. Axiom, developed by a software company called Verse AI, is equipped with prior knowledge about the way objects physically interact with each other in the game world. It then uses an algorithm to model how it expects the game to act in response to input, which is updated based on what it observes—a process dubbed active inference.

The approach draws inspiration from the free energy principle, a theory that seeks to explain intelligence using principles drawn from math, physics, and information theory as well as biology. The free energy principle was developed by Karl Friston, a renowned neuroscientist who is chief scientist at “cognitive computing” company Verses.

Friston told me over video from his home in London that the approach may be especially important for building AI agents. “They have to support the kind of cognition that we see in real brains,” he said. “That requires a consideration, not just of the ability to learn stuff but actually to learn how you act in the world.”

The conventional approach to learning to play games involves training neural networks through what is known as deep reinforcement learning, which involves experimenting and tweaking their parameters in response to either positive or negative feedback. The approach can produce superhuman game-playing algorithms but it requires a great deal of experimentation to work. Axiom masters various simplified versions of popular video games called drive, bounce, hunt, and jump using far fewer examples and less computation power.

“The general goals of the approach and some of its key features track with what I see as the most important problems to focus on to get to AGI,” says François Chollet, an AI researcher who developed ARC 3, a benchmark designed to test the capabilities of modern AI algorithms. Chollet is also exploring novel approaches to machine learning, and is using his benchmark to test models’ abilities to learn how to solve unfamiliar problems rather than simply mimic previous examples.

“The work strikes me as very original, which is great,” he says. “We need more people trying out new ideas away from the beaten path of large language models and reasoning language models.”

Modern AI relies on artificial neural networks that are roughly inspired by the wiring of the brain but work in a fundamentally different way. Over the past decade and a bit, deep learning, an approach that uses neural networks, has enabled computers to do all sorts of impressive things including transcribe speech, recognize faces, and generate images. Most recently, of course, deep learning has led to the large language models that power garrulous and increasingly capable chatbots.

Axiom, in theory, promises a more efficient approach to building AI from scratch. It might be especially effective for creating agents that need to learn efficiently from experience, says Gabe René, the CEO of Verses. René says one finance company has begun experimenting with the company’s technology as a way of modeling the market. “It is a new architecture for AI agents that can learn in real time and is more accurate, more efficient, and much smaller,” René says. “They are literally designed like a digital brain.”

Somewhat ironically, given that Axiom offers an alternative to modern AI and deep learning, the free energy principle was originally influenced by the work of British Canadian computer scientist Geoffrey Hinton, who was awarded both the Turing award and the Nobel Prize for his pioneering work on deep learning. Hinton was a colleague of Friston’s at University College London for years.

For more on Friston and the free energy principle, I highly recommend this 2018 WIRED feature article. Friston’s work also influenced an exciting new theory of consciousness, described in a book WIRED reviewed in 2021.