Join The Community
The DLSS 5 drama continues, as one employee from Nvidia has shared more insight into the technicalities of how the new iteration of the upscaler will work while you're playing games. As many have suspected since the initial reveal video, DLSS 5 isn't actually using your gaming PC's graphical power to enhance, upscale, or render frames differently. In fact, DLSS 5 only uses 2D imagery and motion vectors as an input... so pretty much exactly how an Instagram filter works.
This news comes by way of YouTuber Daniel Owen, who released a video titled "Nvidia answers my DLSS 5 questions" in which he details an email exchange he's had this week with Jacob Freeman, a DLSS Evangelist at Nvidia.
"DLSS 5 only takes the rendered frame and motion vectors as inputs," Freeman said in an email. "Materials are inferred from the rendered frame".
Essentially, that means there isn't anything happening at a deeper engine, rendering level with DLSS 5's image enhancements. It might rely on an Nvidia graphics card to operate, but there's nothing happening while rendering the existing frames of your games with DLSS5 to make them run more smoothly. It's taking cues from the lighting, geometry, fabric, and face models of a video game scene, blending it through a generative AI model, and filtering out a new image based on what the model detects.
This certainly clarifies a few things Nvidia has said this week in response to the DLSS 5 "AI slop" accusations, including the company's CEO Jensen Huang, who said "DLSS 5 fuses controllability of the geometry and textures and everything about the game with generative AI."
When asked more specific questions, Freeman said that in fact, "The underlying geometry is unchanged. Also worth mentioning this is a very early preview of the tech." he said in his email to Owen.
Owen also asked about how one Starfield character in the initial DLSS 5 trailer seemed to have been outright changed by DLSS 5, including hair being rendered where it wasn't in the original game image. "It's painting a 2D picture over the 2D output frame the game actually did... and you won't see the one the game actually made, you'll see the generative AI interpretation of it", Owen observed.
Personally, my biggest concern with DLSS 5, like a lot of video game fans, has been that it seems as though Nvidia outright wants to change the art style and artistic intent of a lot of the games it's showcasing DLSS 5 with. When asked about this, Freeman said:
"Developers will have detailed controls such as intensity and color grading. Artists can use these controls to adjust blending, contrast, saturation, and gamma, and determine where and how enhancements are applied to maintain the game's unique aesthetic."
Owen, talking in response to these questions in his video, talks about the importance of what's not being said in these responses from Nvidia.
Yes, Freeman can say that developers and artists can finetune certain aspects. Essentially, it sounds like developers will have the image adjustments you'd expect from Canva, Instagram, or other services that use image filters. But what he isn't saying here is that game developers will be able to "reprompt" DLSS 5's AI model to enhance things on their own artistic terms. Basically, the way DLSS 5's AI sees and wants to enhance an image seems to be an "on" or "off" job.
"Developers can also mask specific objects or areas to be excluded from enhancement. We continue to talk to developers to understand all the ways they would like to control the technology. Ultimately, we see the NVIDIA DLSS 5 as a tool for them to achieve their artistic vision rather than be limited by the capabilities of traditional real-time rendering," he said.
For more detail, the full video where Owen goes into depth about the questions he asked is linked below.
For more on PC components, check out the best CPU for gaming, the best RAM for gaming, and the best SSD for gaming.
