How important will ray tracing be for Sony's PS5 and Xbox's Project Scarlett?

Ray tracing has been getting a lot of attention recently. Starting with the launch of Nvidia's RTX cards last year – the first graphics cards capable of real-time ray tracing – gaming and the emerging technology have become entangled in a huge way. With confirmation that the PS5 will definitely support ray tracing, and with Microsoft's next offering heavily rumored to as well, it's been cemented as an emerging technology that will shape the conversation around next generation gaming and graphical advancements in the years to come.

What remains to be seen, however, is just how big of an impact real-time ray tracing will actually have and how widely it will be adopted by developers for PS5 and Project Scarlett games. Will the adoption of ray tracing be relegated to high-end PCs or will it arrive at launch on the next wave of next-gen consoles? Some of these questions will only be answered when we've got a more complete picture of just what the next cycle of consoles will be packing, but we can certainly proceed with some healthy speculation with the tidbits currently on our plate.

Ray tracing: a brief refresher

Before we launch into the meat of the discussion, let's quickly cover what ray tracing actually is, and why it's important for real time rendering in video games. Ray tracing is all about modeling light, literally drawing rays through a scene to better understand how light sources interact with objects and generate shadows, reflections, and occlude a scene. It impacts not only what's visible in a scene but what can be reflected in it from off-camera sources, and how lightly or darkly shaded every surface should be. 

There are a number of ray tracing techniques in the modern game developer's tool kit, some of them much more noticeable than others, all of them impacting performance differently. Ambient occlusion, for instance, the process of modeling the greyish shadows in tight spaces around and beneath an object, is a fairly subtle effect that doesn't require a ton of computational horsepower. By contrast, global illumination attempts to simulate how light plays across every surface in a scene, casting rays on a per-pixel basis and calculating where they collide and interact with every visible object. It's a much more dramatic effect, and can really gum up performance, particularly on hardware that lacks an RTX's cards dedicated ray tracing resources. 

Ray tracing for sound works in a similar way. Rays that represent sound waves are bounced around a scene in a way that emulates how real sound travels and the results are calculated and rendered in real time. Like graphical ray tracing it can be implemented in a subtle way to color the sound of a sequence in an unobtrusive way, or much more robustly, to try to get as close as possible to the way a scene would actually sound in the real world.

What ray tracing means for PS5 and Project Scarlett?

Sony and AMD have confirmed that the PS5 will be powered by a Navi GPU and Zen 2 CPU, AMD's latest in graphics and processor microarchitecture. While details about the actual performance ceilings for either component are still extremely muddy, we can assume that they'll be roughly equivalent with contemporary mid-to-high end PC performance. So what kind of ray tracing effects will developers be able to squeeze out of that level of hardware?

Early on, developers who wanted to implement ray tracing had a hard choice to make: either be so subtle as to be nearly imperceptible, or be more significant and risk crippling performance. Even on high end RTX cards, we saw some fairly large frame rate dips when toggling on ray tracing in games like Battlefield 5. But in the years since game studios started getting their hands on real time ray tracing, we've seen some impressive optimizations, and we're now seeing a much larger stable of game engines with native RT support, including indie behemoth Unity and Epic's popular Unreal Engine. So while the ray tracing we see around the launch window of the next consoles may be fairly minor, improvements on the software side will steadily make more dramatic implementations possible. 

I expect this will be especially true of ray traced sound, which is still a fairly new concept and one that hasn't endured nearly the same level of scrutiny as its graphical counterpart, and as a result hasn't had the same level of resources tossed down it's hungry gullet either. Because consoles are typically positioned in the context of a larger entertainment setup, I expect to see the possibilities of ray traced sound and surround sound paired to impressive effect. The promise of realistic echoes, hearing the ominous footsteps of an opponent creeping up behind you while you crouch in the shadows, could finally become a reality, especially important in VR where presence and immersion are so critical.

Properly modeling light is one of the remaining obstacles to approaching photo-realism in games, and ray tracing is looking more and more like the avenue that will get us over the hurdle. Will we see anything like genuine photo-realism in the upcoming console generation? I doubt it, but I wouldn't be surprised if ambitious software engineers were approaching it on PC hardware at the tail end of the next cycle. 

For ray tracing, this console generation will be largely about learning and optimization. We'll see increasingly more convincing occlusion and shadows as developers find ways to introduce them without hamstringing performance, and some truly beautiful effects by way of water caustics (the sparkling effect of light refracted in, for instance, the crest of a breaking wave) and multi-surface reflections/scattering. We may not quite achieve photo-realism, but I expect a generation replete with jaw-dropping wow moments as the artists populating the game studios of the world start to properly understand and harness the power of ray tracing.

Looking forward to the PS5 and Microsoft's Xbox Project Scarlett? Be sure to follow the GamesRadar E3 2019 hub to see all of the new hardware reveals and announcements.