It isn’t sexy, but stats and data are the lifeblood of modern video games. Developers are using telemetry and player feedback more and more to shape updates, DLC and even future iterations of their games. The team at Ubisoft Montreal, who oversee the Assassin’s Creed series, are especially proficient at it. Now, at first glance it seems like the gathering of so much player info is some kind of Orwellian nightmare. How dare the Assassin’s Creed team spy on me while I’m randomly shooting howler monkeys in Black Flag (and secretly liking it)? Well, they are watching us, but that’s actually a good thing. Let me explain… no, even better, let them explain.
This whole feature came about as I started pondering one curious little mechanic in Black Flag, which allows you to rate each mission after you’ve completed it. You can score them from 1 to 5 stars, or choose not to bother. It’s all perfectly optional. The game then merrily sends your feedback away to some chaps at Ubisoft. But what do they actually do with the information? And is that the only data they’re collecting?
I asked Jonathan Dankoff, User Research Project Manager for the Assassin's Creed brand, to explain. “We collect a lot of data through telemetry, from all our online players” he says. “The mission rating system is just the tip of the iceberg, on top of that information we are also able to know everything from how many times you fired a mortar cannon or used a smoke bomb to whether you skipped a cut-scene and the language your subtitles are in.”
Hang on--every time I used a smoke bomb? Or a pistol? So you know about the monkeys? All of them? I’m so, so sorry about that. “All of the data is of course totally anonymous,” he continues. “It’s collected and then analysed in aggregate; we have no interest in zeroing in on a single player, but rather looking at groups that act similarly and then trying to better understand their behaviour in game.” Ah. Ok, ignore all that stuff about the monkeys then. That was definitely someone else.
Dankoff continues, kindly ignoring my barbarism. “In the past our telemetry could only collect things players "did" in the game, but with the mission rating system we now have the opportunity to match their qualitative feedback to their actions, which gives us much deeper insight into not only how they play, but also what they like.”
Ah. Now, that’s the big difference here. While raw data can only tell developers how we played, the mission feedback actually tells them how we felt when we were playing. And that’s huge. In a game like Assassin’s Creed you need to get from A to B progress the story missions. Now, getting from A to B might involve killing someone, following them, sinking a ship… whatever. Now, you may behave in a certain way when you’re doing this--and the raw data shows this.
Oh, so you stabbed 11 men while you were eavesdropping on a Templar conversation. Now, does that mean that you’re a violent player? After all, you could have used stealth to avoid all the guards and just listened to the conversation anyway. Or, are you just sick of eavesdropping missions, and you felt that killing guards was a way to guarantee that you didn’t get spotted, and therefore need to repeat the mission. Hmm… well, if you tell the developers you don’t like the mission by awarding it 1-2 stars at the end, they can extrapolate this info and measure it against the telemetry data. In other words, they’ll know you’re sick of eavesdropping.
Dankoff gives another example: “Here's a fictional example: picture a mission where players have the option to play either stealthily or with a more direct attack. In the past we could count the number of stealth assassination versus kills and get an idea of how many players were completing the mission in each way, but not go much further than that. Now with the mission rating system, we can split those two categories of players and see which ones had a better time in the mission.”
Ok, now pay attention, because here comes the clever bit: “Future designers can use this information to either improve the option that got lower scores to make it as much fun as the other, or gently push the players into the more fun option. The idea isn't to guide their design, but to give them as much powerful and actionable data as possible to help them make the most informed decisions they can.”
To be clear, then. By using the feedback system at the end of each mission, you are helping Ubisoft to shape the future of Assassin’s Creed. You’re not dictating it like some angry internet troll; you’re giving pointers to Ubisoft like a civilised human being, telling them how to make a better game next year. Or whenever the next Assassin’s Creed comes out, which is definitely next year. Just putting that out there.
It isn’t just restricted to Assassin’s Creed either. Thanks to a healthy collaborative culture between studios at Ubisoft, the data gets reviewed and shared with other teams. So, in theory, the way you play Black Flag could have implications for Watch Dogs or even Beyond Good & Evil 2 (try not to kill too many pigs in AC4, yeah?).
Dominic Butler, Team Lead Level Design on Assassin's Creed IV: Black Flag, explains: “The teams find the access to this level of detail fascinating so for sure we try and soak up as much of it as possible but we have to draw the line somewhere. The telemetry tells us what they did and how they did it and coupled with the new ratings system we can see if they enjoyed themselves while doing it.”
“These details are put together with our own findings and our own internal reviews and post-mortems to form a knowledge base resource for the upcoming projects. The teams on those projects then have those resources available to answer questions that would otherwise be impossible to know (e.g. extent to which players explored areas off the beaten track, how they felt about a new game type, etc).”
So don’t get upset about how modern games take your data: at least, not the game-related stuff. It isn’t all designed to target ads more effectively into your eyes. It’s actually pretty useful. Now get out there, grab all the sea shanties you can find, leave some stinking feedback on the over-used ‘follow the Spaniard’ missions, and we’ll all have an awesome, new Assassin’s Creed game to play in 2014…
