More human than human? How the future of video game AI will change the way that we play

An odd thing happened during the development of Assassin's Creed Valhalla. While game director Eric Baptizat was testing a build, he noticed that he was being followed everywhere by two non-player characters. No matter where he went, no matter what he did, these warriors would be there. It seemed that some quirk in Ubisoft's MetaAI system, which gives NPCs persistence and purpose in a game world, had made them zealous disciples. Getting a little frustrated, Baptizat fast travelled to the other side of the country to get rid of them. Ten minutes later, they showed up. They'd hiked across England to reach him – their leader and hero. Nobody designed that to happen, but as an unintended behavior, it tells us a lot about where artificial intelligence in video games is today and how it needs to evolve in the future.

In some ways, video game AI has not evolved greatly over the past decade – at least in terms of the way non-player characters act and react in virtual worlds. Most games use techniques such as behavior trees and finite state machines, which give AI agents a set of specific tasks, states or actions, based on the current situation – kind of like following a flow diagram. These were introduced into games during the 1990s, and they're still working fine, mainly because the action-adventure games of the last generation didn't really require any great advances in behavioral complexity. 

Most NPCs simply patrol a specific area until the player interacts with them, at which point they try to become a more challenging target to hit. That's fine in confined spaces, but in big worlds where NPCs have the freedom to roam, it just doesn't scale. More advanced AI techniques such as machine learning – which uses algorithms to study incoming data, interpret it, and decide on a course of action in real-time – give AI agents much more flexibility and freedom. But developing them is time-consuming, computationally expensive, and a risk because it makes NPCs less predictable – hence the Assassin's Creed Valhalla stalking situation. 

However, as open-world and narrative-based games become more complex, and as modern PCs and consoles display ever more authentic and detailed environments, the need for more advanced AI techniques is growing. It's going to be weird and alienating to be thrust into an almost photorealistic world filled with intricate systems and narrative possibilities, only to discover that non-player characters still act like soulless robots.

The human city

This is something the developers pushing the boundaries of open-world game design understand. Ubisoft, for example, has dedicated AI research teams at its Chengdu, Mumbai, Pune, and Montpelier studios, as well as a Strategic Innovation Lab in Paris and the Montreal studio's La Forge lab, and is working with tech firms and universities on academic AI research topics. 

A practical example of all this is Watch Dogs Legion, which has a good claim on being the first truly next-generation open-world adventure. To create an authentic London filled with living, highly individual inhabitants, the studio developed its Census system, which builds detailed biographies and daily routines for any NPCs you interact with, including relationships with other characters in the world – so if you run over a pedestrian one day, their grief-stricken brother or best friend might come after you the next. 

"On top of that, we have a group behavior system that generates branching events, like crimes and police ID checks," says Martin Walsh, Watch Dogs Legion's technical director of AI and gameplay. "If you happen to see someone in trouble, you can help them out, and this will also cause them to like you personally and DedSec as a whole. We also cast people you have interacted with before into supporting roles in future missions. The characters in Watch Dogs: Legion have lives beyond what you initially see, and if you choose to intervene in their lives it will have lasting consequences."

For Ubisoft, this is just the first step in an ongoing process. "The biggest challenge for AI is to mimic what is perhaps the most complex and mysterious capacity of the human brain: Imagination," says Julien Desaulniers, the programming team lead of AI and Gameplay on Assassin's Creed Valhalla. "Having AI generate narrative content is taking this to a whole new level, one that not even all human beings can perform well."

What he and Walsh foresee is a new generation of AI agents that can have more of an active, intelligent role in the game narrative, perhaps generating new missions and side-story elements on the fly. This will require a combination of emerging AI technologies, which developers are only beginning to grapple with. One example is natural language processing (NLP), a type of AI program that simulates written or spoken human communication – in other words, it writes or (in combination with real-time speech synthesis) talks like a person. 

The use of NLP in games would allow AIs to build human-like conversational elements and then speak them in a naturalistic way without the need for pre-recorded lines of dialogue performed by an actor. Combine these with AI-assisted character animation, which a lot of studios are now using to augment motion-capture and make characters more naturally responsive to the environment, and you might have NPCs that can think, talk, act, and plan like real people. 

Teaching a game to act

We're a little way from this just yet, but it's coming. "The capacity of NPCs to say things that were not planned for, not recorded in advance, is one aspect of tech we are still not able to master," says Desaulniers. "While we do have systems working for us that can deliver lines with reasonably convincing performances in terms of virtual acting, we are not quite 'there' yet to do this in real-time while the player interacts with the game-world. Overall, this is a very exciting challenge for us, with tremendous potential for tech developers and storytellers alike." Walsh agrees: "In the near future, voice and animation synthesis will be good enough to allow us to create characters and stories without having to mo-cap and record all of the data. This would allow for much faster content generation, which could really change how we create characters, and what types of stories can be told."

There are companies already working toward this. UK based start-up Sonantic has developed an artificial voice technology, a kind of virtual actor, which can deliver lines of dialogue with convincing emotional depth, adding fear, joy and shock, depending on the situation. The system requires a real voice actor to deliver a couple of hours of voice recordings, but then the AI learns the voice and can perform the role itself. Sonantic is already working with a range of studios, including Splash Damage and Obsidian, and they're mostly using it during the development stages to test in-game dialogue, but in the future, the tech could be used to make in-game dialogue much more contextual and reactive. "Soon voices will be running live, dynamically in the game," says co-founder and CEO Zeena Qureshi. "If your character is out of breath, they will sound out of breath. If you wronged a character in a previous level, they will sound annoyed with you later." 

This advancement of AI into the development process isn't about replacing game writers and designers, though – Ubisoft isn't just going to hand over Far Cry 7 to Skynet. But it could lead to huge changes in the way stories play out and the way we interact with NPCs. Perhaps we will finally do away with intrusive non-interactive cutscenes, with intelligent NPCs able to deliver story moments and exposition during the game, at moments where each individual player needs to experience them, depending on what they've already done and seen. We may also see the back of fixed, pre-written dialogue trees in RPGs such as Mass Effect and The Witcher – an AI agent with an NLP component could engage in realistic conversations with players, either via voice or onscreen text, making the prospect of romancing Garrus much more involving. 

The use of machine learning techniques could also make NPCs more reactive to player actions. Think about those in-game moments where characters are delivering long speeches while we're idly exploring the room, jumping on tables, picking up pot plants, getting really close to the speaker… Until now, that character just keeps talking obliviously, but if they're an AI agent with NLP and AI-assisted animations, they could react. "We will definitely see games where the NPC will say 'why are you putting that bucket on your head?'" says AI researcher Julian Togelius. "This is something you can build-out of a language model and a perception model, and it will really further the perception of life.  

"I mean, how do you find out if you're talking to a chatbot or a customer services employee when you're online? Well, you do it by forcing them to acknowledge and respond to something you said or did – this is currently underused in video game design and I think will be used much more in the future, so it comes through that they understand the player in some way. It's the illusion of life."

Togelius, who is working on an unannounced video game project that utilizes these technologies, is excited by the prospect of chatty autonomous agents. "You would be able to use freeform inputs and the outputs would be different every time, which would be much more believable. You could also have a Mass Effect style interface, where the player simply stipulates 'say a threatening thing', and if you asked the same thing too many times in a row, the language model could get really annoyed at you and tell you to leave."

Language lessons

Several AI coders we spoke to were particularly excited by GTP-3, the highly advanced NLP model developed by San Francisco-based laboratory OpenAI, owned by Elon Musk. GTP-3, which can read enormous amounts of text and then generate its own written content, has already been employed to craft magazine and newspaper articles. So might we see an open-world adventure with GTP-3 equipped NPCs in the future? "You get very good varied results from GTP-3, but there is a control problem, and Open AI is very aware of that and of the potential for PR disasters," says Togelius. In short, what if you get an AI game character who decides to use racist or sexist slurs, for example, or just goes totally wild in the game world?

It's something Monolith Productions, the team behind Middle-earth: Shadow of War, is aware of as it builds on its seminal Nemesis System, which creates real-time rivalries between the player and NPCs. "General technological advances in areas like natural language processing and text generation are fascinating," says Michael de Plater‏, creative VP at the studio. "There's a wonderful blog by Janelle Shane called AI Weirdness with great examples of the insanity AI can generate, which speaks to its strengths but also its limitations."

"Interactive Fiction is constantly fascinating, and Emily Short has a brilliant blog on Interactive Storytelling and AI," de Plater‏ continues. "As far as recent games, the reactivity and relationship building in Hades by Supergiant Games was brilliant. The other constant inspiration is tabletop roleplaying; we're basically trying to be great digital Dungeon Masters."

The answer to rogue AIs may be a tightly controlled vocabulary and a few pre-written prompts. "You could have freeform conversations, but you could also combine this with bits and pieces of scripted text," says Togelius. "I fully expect that within a year someone else will have essentially implemented GTP-3 in a game."

You are being watched

Another area of AI that's likely to become more important in the future is player modeling, in which player actions within a game are studied and memorized by the AI system. Of course, we've seen many games that feature enemies who learn player tactics and alter their own accordingly – the fighting game genre is full of examples – and we're also used to enemies that call out your position in the game world. But we also love games with characters that simply notice us – like the NPCs who comment on your bloody clothes in Red Dead Redemption 2, or the bartenders in Hitman 3 who ask what the hell you're doing hiding behind the drinks fridge. 

Togelius sees much more advanced examples of this in the near future. "If you have a good idea of what a player might do or where they like going in the world, then there are a lot of story patterns that can be instigated as quests in many different ways," he says. "Say you want to give the player a fetch quest, you could alter it depending on what you expect that player to do... if it's the kind of player who will explore far away islands, you can decide that's where to send them. There are a lot of light quests you could re-balance, so there is sneaking here or combat there, based on the player's past experiences. You can still design the main questline, but then how that manifests in the game can be changed based on what you know about the individual player." 

We've seen lots of games, like Fable, with simple morality systems where the world treats you differently if you've been good or evil. But modern AI could add greater nuance and complexity, with recognition manifesting in more profound ways than reputation points. "If there's a particular symbol that you always get a blacksmith to engrave on your armor, perhaps other characters will get the same symbol because they've seen it on you. Also, when you're hunting monsters, you may always use the same technique of sneaking up behind them and shooting them with a poison arrow, then you look around and realize NPCs are now doing it the same way. They learn your movement patterns and replicate them. This is the kind of stuff we do need modern machine learning to enable," says Togelius. 

Player modeling could also combine with NLP in future open-world adventures, so you could have people in the game world retelling stories to each other about the things you've done. Rumors and gossip will be exchanged between NPCs, as will myths and legends. Imagine arriving in a village in The Witcher 4 to find a minstrel singing songs about your last dragon encounter or the very specific way you dealt with the Bloody Baron.

Movement and mimicry

Like Ubisoft, Electronic Arts is heavily involved in AI research. The publisher has central teams such as EA Digital Platform and a dedicated research division, SEED, working with advanced AI technologies. Like other developers, one of its major interests is the use of AI to augment asset creation, such as extremely natural and detailed textures, but also more authentic and reactive character animation. The ability to combine mo-cap animations with real-time responses is going to be vital to make sure characters interact in a realistic manner with complex game worlds, rather than running into doors or loping awkwardly up staircases. 

"Animation blending and motion matching is now being handled by machine learning," says Tommy Thompson, director of the consultancy, AI and Games, and one of the foremost experts in video game artificial intelligence. "It figures out the best blend points between animations and re-targets those animations based on different contexts. So a character would be able to sit down in 10 different chairs, using the same animation, with the AI engine able to blend the movements seamlessly."

Right now, EA is investigating methods of using deep learning to capture realistic motion and facial likenesses directly from video instead of having to carry out expensive and time-consuming motion capture sessions. "This is something that will have a big impact in my opinion, especially for sports games in the future," says Paul McComas, EA's head of animation. "This motion data will allow us to cover more and more gameplay situations, and it will also appear more natural because we will get animation data from real athletes 'in the wild', if you will, as opposed to the vacuum of a motion capture studio." 

"This means more realism, especially for contact sports, as well as more true-to-life signature moves and behaviors from big personalities. Being able to capture new material from video will also make us more reactive, like, for example, quickly introducing a new move from an athlete or even create scenarios based on real-life events, soon after they happened," says McComas.

The publisher is also looking into using a combination of reinforcement learning (where an AI learns through trial and error), and imitation learning (where it learns by watching and mimicking human actions), to create FIFA and Madden players that accurately behave like their real-life counterparts. But there are wider applications of this. If we can train AIs to behave like real football players, then we can train them to behave like superstar pro gamers and streamers too. 

"Think about this in context of Fortnite," says Togelius. "Every player could get the chance to play against a virtual Ninja – there's a whole subset of the game's audience who would kill for that opportunity. You could take the best crew of Sea of Thieves streamers and create an AI team – although that's really difficult because getting four human players to play Sea of Thieves effectively is hard enough, let alone four AIs!"

EA is also interested in using machine learning to enhance user-generated content. "It will make it easier for users to create avatars that look like themselves with just a phone, capture gestures and facial expressions, as well as offering smarter tools to create level and assets intuitively," says Fabio Zinno, senior software engineer at EA. "I also think that Natural Language Processing will help gamers with disabilities, for example, getting text from speech."

The next generation of open-world adventures will react to the player, with missions that skew to your actions and preferences, and with characters who simulate empathy as well as aggression, and who come to know you as an individual. Those two NPCs that Ubisoft's Eric Baptizat encountered, who tracked him across the country, not because the game told them to but because they wanted to, may actually tell us a lot about where games are going. Reactive, observant, a little unpredictable at times. It's kind of ironic that all this incredibly advanced research and development is going into something so seemingly straightforward: the future for games is to become more like us.

Many thanks to Tommy Thompson who provided a huge amount of technical background information on AI for this article. See his excellent YouTube channel, AI and Games, here.