The machine learning architecture used with Arc Raiders enemies effectively leads to a feedback loop where the Arc seek out players because they receive "rewards" for doing so, and the multiple neural network "brains" controlling these things absolutely love to be rewarded. In fact, they exist to be rewarded.
This is an extremely quick-and-dirty version of the highly technical GDC talk that Martin Singh-Blom, developer Embark's machine learning research lead, conducted this week. I was in the crowd furiously taking notes, and Singh-Blom's description of the Arc sounds fittingly sci-fi with a very basic understanding of the technology.
For Arc Raiders' purposes, machine learning was used to teach Arc how to move in convincing and dynamic ways. Through a chain of checks and balances gauged against recognition and affirmation, and with those systems parked comfortably in the cloud, the Arc are able to identify good and bad outcomes and determine how to reach them. Good actions increase the reward value and bad actions decrease it.
Arc - or at least the ones using machine learning, which are generally the multi-legged ones like Bastions and Leapers - earn reward points through tasks like moving toward a target, staying upright, and orienting themselves to a target. Points may be lost by using too much energy (within the game's physics) or putting their limbs at bizarre angles, which keeps them from going ballistic in a mad dash.
Crucially, Arc are also rewarded for emulating conventional animations in their movements. Embark has animators who whipped up examples of how Arc would move in a perfect world, but rather than directly puppeteering the Arc, which would come with huge limitations in a game with a dynamic sandbox, those animations are often used as a critical reference in machine learning. The more closely the Arc mimic the assigned animation, the more rewards they earn, meaning those base animations can be adapted to different conditions in the greasy, clanker hands of the Arc.
In a cheeky summary of this, Singh-Blom shares a line from an old development progress video showing a proto-Arc hobbling around an uneven field. "I learned how to walk on my own," the machine claims. Elsewhere, the bot, still stumbling but walking more steadily in every new video clip, warns, "Payback is coming. Eventually."
Arc do use more traditional video game AI behavior trees, but they are also controlled by machine learning "brains" that have been trained on different materials to suit different tasks. Think of each brain as a specialized pilot for the Arc, except one made of behavioral directives rather than puny flesh. The brain's goal - and there are several brains that may take control regularly depending on the situation, from the "patrol walk brain" to the "pursuit brain" - is generally to earn reward points, at least in the context of locomotion.
The result is a cast of uncanny, motivated-looking AI entities that can overcome unpredictable situations - like moving over or around debris at sharp angles, while weathering grenade fire, or with missing limbs - as they pursue you. It took Embark years to get a single Arc up and running - quite literally - in an effective way, but once the system was built, the devs were able to apply it to a whole fleet of Arc relatively quickly. Singh-Blom suggests there's still plenty of space to improve the technology as well, meaning future Arc, or even the enemies we already know, could become significantly smarter.
Arc Raiders turns down electromagnetic storm lightning despite some players preferring the chaos, as Embark promises compensation for folks impacted by recent server issues.
