To test Starfield, Bethesda gave the team a build that was "basically done" last year and said "this should be the game you're playing over the holiday"

Bethesda boss Todd Howard says Starfield was "basically done" by holiday 2022, and to help test the mammoth RPG, virtually everyone on the development team got a build to play on their own Xbox consoles and PCs at home.

In a new Game Maker's Notebook interview with Insomniac Games CEO Ted Price, Howard discusses the final year of Starfield's development and the team-wide polish that came of it. Price asks how you test a massive RPG like Starfield, and Howard says "you take a long time and a lot of people."

"Our QA staff is awesome here, and the whole team tests," he continues. "One of our goals last holiday was – everybody went home for holiday break and the game was basically done. There were bugs, but here's a full game, you can play it – we flighted it on the retail Xbox. You can play it at home on your Xbox and your PC, and this should be the game you're playing over the holiday. The whole team, we tested the game really all year just playing it all the time, tweaking it, fixing I don't want to say how many bugs that our games create. We had a lot of QA help.

"It does help understanding what kind of bugs your game can create and then seeing patterns, using systems to go through the data. We do use some automated systems that run through every space in the game and then get a report. Here's where it was slow, here's where it crashed, here's where something else went wrong. We can focus more on the systemic gameplay. Here's 10 ways to break this quest. 

"We're really happy with where we landed on this release. Given the scale of the game, I think the team did an amazing job. It's not perfect by any means. There are things we're obviously fixing and are gonna continue to fix. But given the scale of it, where we ended up both on a bug that would block you from playing and performance, particularly on the consoles, we ended up in a pretty good place I think."

Later in the interview, Howard explains that Bethesda always wanted a long testing period for Starfield but didn't totally foresee the nine-month window it ended up with after the RPG's multiple delays. "I want to say it was like six months," he says. "Six months out, we want to be playing a game that feels done. Sometimes you feel that you launch a game and you're catching up on things. But no matter what you do, it doesn't equal putting it out and 10 million people play it. Anything you do internally doesn't equal that, but you can put yourself in a much better position."

Another 2023 mega-release, The Legend of Zelda: Tears of the Kingdom, was also basically finished a year before its release. Series producer Eiji Aonuma said that the game was pretty complete when its year-long delay was announced, giving Nintendo time to focus on polish and QA. This is a rare luxury for games of this or indeed any scale – time tables routinely force developers to ship games before they may fully want to – but it's clearly paid off in both cases. Even without Bethesda's, shall we say, unique standards, Starfield isn't all that buggy. 

Howard's description of the Bethesda team playtesting Starfield at home, and seemingly over a holiday break, does raise an eyebrow. But from the way Howard talks about it, it sounds like it was more about integrating QA into regular work-from-home time: "Even if you're working at home you're usually on your PC. We knew given the scale of the game that's where we wanted to be, so we made it a goal for this project, to have that much time to be playing it and polishing it."

Even so, game development has long since lost the benefit of the doubt when it comes to labor. I've reached out to Bethesda to ask about how this take-home QA was scheduled. 

In the same interview, Howard reveals that exploring planets in Starfield used to be a lot more brutal before Bethesda "nerfed the hell out of it."