It is, by now, a very familiar story. A studio spends years beavering away on a game, staging carefully-managed demos and releasing beautifully-edited videos that highlight their work-in-progress project in the best possible light. And then this game finally launches, and the world realises that this isn’t the choreographed experience they’d glimpsed in trailers and tradeshows. It’s imperfect. It’s flawed. And it contains bugs. That’s usually when the online recriminations begin, a barrage of forum posts and Tweets calling out the developers for laziness, or bemoaning the fact that a money-grabbing publisher pushed out an unfinished game for a quick buck rather than choosing to hold the title back for more development time.
But this perspective, while understandable, doesn’t quite reflect the realities of game development. Creating interactive software is an incredibly complicated process that often involves teams of hundreds of genius-level artists, coders and animators all attempting to co-ordinate their efforts towards a singular vision. As such, it’s inevitable that things will go wrong from time to time.
And if there’s a perception that games are buggier now than they’ve ever been before, that might just be because it’s true. After all, games are now vastly more complicated than at any other point in the medium’s history. For proof you need only look at the average team size for a triple-a video game – according to Epic’s Mark Rein, only 20 to 30 people were involved in the development of the original Gears Of War at any one time. Gears Of War 4, meanwhile, involved roughly 330 people at the peak of its production phase.
The consequence of that inflation of headcounts is that the codebase for your typical blockbuster is simply enormous. So large, in fact, that it would be impossible for any single person to know what’s going on with every aspect of the game.
Show and tell
But the presence of more bugs is only half of the problem, as Monomi Park co-founder Nick Popovich explains to us. “Games have become so complex and players are also so much more educated on the technical side of things. That means that we both ship more bugs and players are better at spotting them.” Fifteen years ago, a player might have encountered a glitch and thought nothing of it. Today, they’ll not only know that they’re looking at a bug, they’ll also have the tools and the know-how to share it to YouTube immediately. Within days of a flagship game’s release, you can bet there’ll be dozens of compilations bringing together every glitch and quirk it has to offer, further heightening the perception that the game is riddled with bugs.
Another consequence of team sizes spiralling into the hundreds is that publishers and developers have to be extremely efficient about where their staff are deployed. After all, if your studio comprises 150 staff, it would be financially ruinous to have all of your concept artists and level designers twiddling their thumbs once the game enters its polishing phase. As such, full-time studio staff are generally moved onto new projects as soon as their contribution to a project is concluded.
So, by the time it becomes clear that a game needs more development time to squash all of the bugs thrown up by testing, many of the relevant studio staff will have already moved on to other projects. In these cases, delaying the game won’t just cost an enormous amount of money and compromise the publisher’s marketing plan – it could even cause delays to entirely separate games, as studio staff are pulled away from other titles.
Bugspray
And all that’s to say nothing of the fact that a bug-free launch is basically a fairytale. Once a game passes from the hands of its developers into the lap of the quality assurance team, they’re likely to discover hundreds of bugs of varying levels of severity – from game-breaking glitches, to small technical issues that only arise under very specific circumstances. The QA team will then report back to the developers with an exhaustive catalogue of every issue found during testing.
But what happens next might surprise you, because the development team doesn’t simply set about fixing every bug that’s been discovered. Instead, it will usually categorise the issues in order of severity and repeatability – with catastrophic glitches that arise under repeatable circumstances being the most urgent candidates for a fix. If a bug is not severe or will only arise in very unusual circumstances, many studios will choose to just leave it in the game.
Why would developers do this? There are a number of reasons, the first of which is that time and money are both finite resources, and developers have to spend both wisely. The second is that bugs are frequently tied to fundamental parts of a game (such as the engine or physics system), meaning that it’s often impossible to address the root cause without running the risk of introducing hundreds more bugs. For this reason, the simplest fix is often the best. Ragdoll physics stop working around a certain item? Better to bin the item rather than diving into the codebase. Game keeps crashing in a certain room? Maybe the room could just be cut altogether.
For independent developers, the team sizes may be smaller but the process is strikingly similar. When we ask Popovich to run us through the process of fixing a bug, he explains that the process always involves multiple parts. “The first is actually identifying the cause, which is what players often mistake for the whole process. When you see a character fall through the floor in a game, it’s actually the symptom of the bug, not really the bug itself. From there you need to dig and discover what’s causing it and that can take a really long time. Then it’s sometimes a hairy situation when you realise the system causing the bug has its tentacles in so many other systems that changing it has a ripple effect and... you see where this is going,” he explains. “So my point is: when you are pointing out bugs to a dev that have existed in the game for a long time, chances are they’re very aware of it and just haven’t been able to determine the cause.”
But even if a developer finds a bug and manages to successfully determine the cause, the pain doesn’t stop there. Fixing a bug can (and often does) introduce more issues, and this is why bug counts don’t always just decline in the run-up to release – they fluctuate. The developer just has to hope that, for every major bug it squashes, only minor bugs are introduced.
Take the original Crackdown, for instance, which launched back in 2007. It’s usually remembered for Agility Orbs and exuberant explosions rather than game-breaking glitches, but speaking several months after the game’s launch, Realtime Worlds’ producer told the Montreal Games Summit that Microsoft’s QA team found a total of 37,000 bugs lurking in the game’s pre-release code. If the studio had committed itself to fixing every one of those bugs before launch, there’s every chance we’d still be waiting for Crackdown today. Instead, the studio did the sensible thing – it decided to ignore more than 3,000 bugs that were deemed ‘low severity’. “I see them when I play the game,” Wilson said, “but people were very forgiving.”
Life’s a glitch
People haven’t been in such a forgiving mood lately, though. Take Mass Effect: Andromeda, for instance – a game that was subjected to scorn and mockery online thanks to a number of technical issues. It would be fair to say that the picture that’s emerged of Andromeda’s development in the months since its release hasn’t been entirely rosy, with several anonymous sources describing periods of intense crunch and rudderless toil. But while these accounts may not make for happy reading, they do provide a certain amount of insight into why Andromeda launched with such obvious technical and animation issues.
The transition from the Unreal Engine (which BioWare used for the original Mass Effect trilogy) to EA’s Frostbite engine was clearly a huge factor, posing a significant challenge to the developer. Frostbite was built for the Battlefield franchise, after all, so while it’s clearly capable of producing gorgeous results, it’s been put together with a fast-paced first-person shooter in mind. As such, BioWare had to build a host of entirely new functionality into the engine from scratch, a process that was both challenging and time-consuming.
And then there’s the issue of Andromeda’s facial animation. At launch, the game’s gurning character models were widely shared and ridiculed online, leading many gamers to criticise BioWare for releasing the game before these issues could be resolved. But the truth is that it would be almost impossible to hand-animate every single conversation in a game as vast, customisable and story-heavy as Mass Effect: Andromeda. As such, studios that specialise in narrative-driven role-playing games often make use of advanced ‘conversation systems’ that can automatically generate simple dialogue scenes from a vast library of poses and gestures. The most important exchanges will then be lovingly spruced up by designers – other scenes may barely be touched by human hands.
All of which is to say that it’s a miracle that any video game ever gets launched at all. After all, these are fearsomely complicated pieces of software that are often created by hundreds of people over the course of many years. And unfortunately, it will often only become clear whether that work has paid off in the very last few months before release. As Nick Popovich explains, even the most well-resourced teams can run into trouble – it’s simply the nature of development. “Just because you can see the bug in action it doesn’t mean that the next step is fixing it. A bug can be incredibly hard to identify what’s actually causing it, even by the most experienced devs with the most resources allotted to them.” The issue isn’t that the publisher is making a cynical grab for wallets, or that developers are being lazy. The issue is that making games is almost ludicrously hard. “So just know that a bad bug bothers the devs just as much as players.”
This article originally appeared in Xbox: The Official Magazine. For more great Xbox coverage, you can subscribe here.
